Endoscope device

ABSTRACT

In an endoscope device, an imaging device sequentially reads pixel signals from at least some of a plurality of pixels row by row during a first period. A control unit causes a light source to generate illumination light during a second period. The second period is at least a part of a period other than the first period. The control unit causes the light source to stop the generation of the illumination light during a third period. The third period is all of a period other than the second period. The control unit causes a switching unit to start switching of an imaging condition during the third period and complete the switching of the imaging condition during the third period.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an endoscope device.

Priority is claimed on Japanese Patent Application No. 2018-082654,filed Apr. 23, 2018, the content of which is incorporated herein byreference.

Description of Related Art

Industrial endoscopes are widely used for observing internal damage andcorrosion in boilers, engines, turbines, chemical plants, and the like.When defects such as damage and corrosion are found, it is necessary toperform switching between countermeasure methods in accordance with adegree thereof. Thus, an industrial endoscope having a measurementfunction of measuring sizes of damage and corrosion has been developed.

An endoscope device disclosed in Japanese Unexamined Patent Application,First Publication No. 2013-105078 includes an optical system for causingtwo optical images of a subject to be formed in a common region of animaging device. Light passing through two optical paths corresponding totwo different viewpoints forms two optical images. Hereinafter, the twooptical paths are refered to as a first optical path and a secondoptical path. The endoscope device includes an optical path switchingmeans for performing switching between two optical paths. The endoscopedevice captures an optical image formed by only light passing througheither one of the two optical paths.

The endoscope device performs switching between two imaging conditionsand acquires two images. The light passing through the first opticalpath forms a first optical image. The first optical image is an opticalimage from a first viewpoint. The endoscope device generates a firstimage by capturing the first optical image. At this moment, the firstimaging condition is implemented. Subsequently, optical paths areswitched. The light passing through the second optical path forms asecond optical image. The second optical image is an optical image froma second viewpoint. The endoscope device generates a second image byimaging the second optical image. At this moment, the second imagingcondition is implemented. The endoscope device measures a shape of asubject using the principle of stereo measurement on the basis ofparallaxes provided in the first image and the second image. The firstimage and the second image are images captured from viewpoints differentfrom each other.

When the tip of the endoscope or the subject moves while the first imageand the second image are acquired, a positional relationship between twoviewpoints changes and a mismatch between a stereo measurement parameter(such as a baseline length) and positions of two viewpoints occurs.Therefore, the endoscope device cannot accurately measure the shape ofthe subject. The endoscope device disclosed in Japanese UnexaminedPatent Application, First Publication No. 2013-105078 alternatelyacquires a first image and a second image. When the amount of positionshift between two first images is less than a predetermined thresholdvalue, the endoscope device determines that there is no movement of anendoscope tip (tip movement) or movement of a subject (tip movement)during a period in which two first images are acquired and performs ameasurement process.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, an endoscopedevice includes a light source, an illumination optical system, anobservation optical system, an imaging device, a switching unit, and acontrol unit. The light source generates illumination light. Theillumination optical system radiates the illumination light to asubject. The observation optical system forms an optical image of thesubject. The imaging device has a plurality of pixels disposed in amatrix and images the subject. The imaging device sequentially readspixel signals from at least some of the plurality of pixels row by rowduring a first period. The imaging device generates an image of thesubject during each frame period of a plurality of frame periods on thebasis of the pixel signals read from at least some of the plurality ofpixels. The pixel signals are generated on the basis of the opticalimage of the subject. The switching unit performs switching between aplurality of imaging conditions so that the imaging device images thesubject. The control unit causes the light source to generate theillumination light during a second period. The second period is at leasta part of a period other than the first period. The control unit causesthe light source to stop the generation of the illumination light duringa third period. The third period is all of a period other than thesecond period and includes the first period. The second period and thethird period are alternately iterated. The control unit causes theswitching unit to start switching of the imaging condition during thethird period and complete the switching of the imaging condition duringthe third period.

According to a second aspect of the present invention, in the firstaspect, the imaging device may read the pixel signals in a time periodthat is less than or equal to half a length of each frame period of theplurality of frame periods during the first period.

According to a third aspect of the present invention, in the firstaspect, the endoscope device may further include an image processingunit configured to execute image processing on a plurality of imagesgenerated during the plurality of frame periods. The plurality ofimaging conditions may include a first imaging condition and a secondimaging condition. The first imaging condition and the second imagingcondition may be different from each other. The imaging device maygenerate a first image of the subject by imaging the subject under thefirst imaging condition. The imaging device may generate a second imageof the subject by imaging the subject under the second imagingcondition. The image processing unit may process the first image and thesecond image.

According to a fourth aspect of the present invention, in the thirdaspect, the imaging device may generate a plurality of at least one offirst images and second images. When the imaging device generates theplurality of first images, the image processing unit may calculate avalue indicating whether or not the plurality of first images aresuitable for image processing. When the imaging device generates theplurality of second images, the image processing unit may calculate avalue indicating whether or not the plurality of second images aresuitable for image processing.

According to a fifth aspect of the present invention, in the thirdaspect, the imaging device may generate a plurality of first images byimaging the subject under the first imaging condition. The imagingdevice may generate a plurality of second images by imaging the subjectunder the second imaging condition. The image processing unit maygenerate a third image by executing a noise reduction process on theplurality of first images. The image processing unit may generate afourth image by executing the noise reduction process on the pluralityof second images. The image processing unit may execute a processdifferent from the noise reduction process on the third image and thefourth image.

According to a sixth aspect of the present invention, in the thirdaspect, the image processing unit may calculate three-dimensional (3D)coordinates of at least one point on a surface of the subject on thebasis of the first image and the second image.

According to a seventh aspect of the present invention, in the fifthaspect, the image processing unit may calculate 3D coordinates of atleast one point on a surface of the subject on the basis of the thirdimage and the fourth image.

According to an eighth aspect of the present invention, in the sixthaspect, the endoscope device may further include a data generation unit.The illumination light may be white light. The imaging device mayinclude a red color filter, a green color filter, and a blue colorfilter. The imaging device may generate a color image as the image ofthe subject. The color image may have information indicating each ofbrightness of red, brightness of green, and brightness of blue. The datageneration unit may generate data in which 3D coordinates of a pluralityof points on the surface of the subject are associated with theinformation corresponding to the plurality of points.

According to a ninth aspect of the present invention, in the sixthaspect, the observation optical system may include a first opticalsystem and a second optical system. The first optical system and thesecond optical system may be disposed on an optical front side of theimaging device. The first optical system may form a first optical imageof the subject corresponding to a first viewpoint on the imaging device.The second optical system may form a second optical image of the subjectcorresponding to a second viewpoint different from the first viewpointon the imaging device. The switching unit may cause light that passesthrough the first optical system to be incident on the imaging deviceand block light that passes through the second optical system under thefirst imaging condition. The switching unit may cause light that passesthrough the second optical system to be incident on the imaging deviceand block light that passes through the first optical system under thesecond imaging condition. The control unit may switch the optical imageto be formed on the imaging device between the first optical image andthe second optical image by controlling the switching unit. The imageprocessing unit may calculate the 3D coordinates using a passive stereomethod on the basis of the first image corresponding to the firstoptical image and the second image corresponding to the second opticalimage.

According to a tenth aspect of the present invention, in the firstaspect, the endoscope device may further include an image processingunit, a data generation unit, a first light source, and a second lightsource. The image processing unit may execute image processing on aplurality of images during each frame period of the plurality of frameperiods. The first light source and the second light source may serve asthe light source. The illumination light may include first illuminationlight and second illumination light. The first light source may generatewhite light serving as the first illumination light. The second lightsource may generate the second illumination light. The illuminationoptical system may include a pattern generation unit configured to givea spatial pattern including a bright part and a dark part to the secondillumination light. The illumination optical system may radiate thesecond illumination light to which the pattern is given to the subject.The plurality of imaging conditions may include a first imagingcondition and a second imaging condition. Under the first imagingcondition, the first illumination light may be radiated to the subjectand radiation of the second illumination light to the subject may bestopped. Under the second imaging condition, the second illuminationlight may be radiated to the subject and radiation of the firstillumination light to the subject may be stopped. The imaging device maygenerate a first image of the subject by imaging the subject under thefirst imaging condition. The imaging device may generate a second imageof the subject by imaging the subject under the second imagingcondition. The image processing unit may calculate 3D coordinates of aplurality of points on a surface of the subject using an active stereomethod on the basis of the second image. The imaging device may includea red color filter, a green color filter, and a blue color filter. Theimaging device may generate a color image as the first image. The colorimage may have information indicating each of brightness of red,brightness of green, and brightness of blue. The data generation unitmay generate data in which the 3D coordinates of the plurality of pointsare associated with the information corresponding to the plurality ofpoints.

According to an eleventh aspect of the present invention, in the tenthaspect, the imaging device may generate a plurality of at least one offirst images and second images. When the imaging device generates theplurality of first images, the image processing unit may calculate avalue indicating whether or not the plurality of first images aresuitable for image processing. When the imaging device generates theplurality of second images, the image processing unit may calculate avalue indicating whether or not the plurality of second images aresuitable for image processing.

According to a twelfth aspect of the present invention, in the tenthaspect, the imaging device may generate a plurality of second images byimaging the subject under the second imaging condition. The imageprocessing unit may generate a third image by executing a noisereduction process on the plurality of second images. The imageprocessing unit may calculate the 3D coordinates of the plurality ofpoints on the basis of the third image.

According to a thirteenth aspect of the present invention, in the thirdaspect, the illumination optical system may include a pattern generationunit configured to give a spatial pattern including a bright part and adark part to the illumination light. The plurality of imaging conditionsmay further include a third imaging condition. The imaging device maygenerate a third image of the subject by imaging the subject under thethird imaging condition. The switching unit may perform switchingbetween the first imaging condition, the second imaging condition, andthe third imaging condition by causing a phase of the pattern of theillumination light to be shifted. A pattern phase under the firstimaging condition, a pattern phase under the second imaging condition,and a pattern phase under the third imaging condition may be differentfrom one another. The image processing unit may calculate 3D coordinatesof a plurality of points on a surface of the subject using a phase shiftmethod on the basis of at least the first image, the second image, andthe third image.

According to a fourteenth aspect of the present invention, in thethirteenth aspect, the endoscope device may further include a datageneration unit, a first light source, and a second light source. Thefirst light source and the second light source serve as the lightsource. The illumination light may include first illumination light andsecond illumination light. The first light source may generate whitelight serving as the first illumination light. The second light sourcemay generate the second illumination light. The pattern generation unitmay give the pattern to the second illumination light. The plurality ofimaging conditions may further include a fourth imaging condition. Underthe first imaging condition, the second imaging condition, and the thirdimaging condition, the second illumination light may be radiated to thesubject and radiation of the first illumination light to the subject maybe stopped. Under the fourth imaging condition, the first illuminationlight may be radiated to the subject and radiation of the secondillumination light to the subject may be stopped. The imaging device maygenerate a fourth image of the subject by imaging the subject under thefourth imaging condition. The imaging device may include a red colorfilter, a green color filter, and a blue color filter. The imagingdevice may generate a color image as the fourth image. The color imagemay have information indicating each of brightness of red, brightness ofgreen, and brightness of blue. The data generation unit may generatedata in which the 3D coordinates of the plurality of points on thesurface of the subject are associated with the information correspondingto the plurality of points.

According to a fifteenth aspect of the present invention, in thefourteenth aspect, the imaging device may generate a plurality of atleast one of first images, second images, third images, and fourthimages. When the imaging device generates the plurality of first images,the image processing unit may calculate a value indicating whether ornot the plurality of first images are suitable for image processing.When the imaging device generates the plurality of second images, theimage processing unit may calculate a value indicating whether or notthe plurality of second images are suitable for image processing Whenthe imaging device generates the plurality of third images, the imageprocessing unit may calculate a value indicating whether or not theplurality of third images are suitable for image processing. When theimaging device generates the plurality of fourth images, the imageprocessing unit may calculate a value indicating whether or not theplurality of fourth images are suitable for image processing.

According to a sixteenth aspect of the present invention, in thethirteenth aspect, the control unit may cause the switching unit to setthe first imaging condition during a plurality of first frame periods.The control unit may cause the switching unit to set the second imagingcondition during a plurality of second frame periods. Each second frameperiod of the plurality of second frame periods may be different fromeach first frame period of the plurality of first frame periods. Thecontrol unit may cause the switching unit to set the third imagingcondition during a plurality of third frame periods. Each third frameperiod of the plurality of third frame periods may be different fromeach first frame period of the plurality of first frame periods and maybe different from each second frame period of the plurality of secondframe periods. The image processing unit may generate a fifth image byexecuting a noise reduction process on a plurality of first images. Theimage processing unit may generate a sixth image by executing the noisereduction process on a plurality of second images. The image processingunit may generate a seventh image by executing the noise reductionprocess on a plurality of third images. The image processing unit maycalculate the 3D coordinates of the plurality of points on the basis ofthe fifth image, the sixth image, and the seventh image.

According to a seventeenth aspect of the present invention, in the thirdaspect, a focus of the observation optical system under the firstimaging condition may be different from a focus of the observationoptical system under the second imaging condition. The image processingunit may generate a third image by synthesizing the first image and thesecond image.

According to an eighteenth aspect of the present invention, in the thirdaspect, the amount of light of the light source under the first imagingcondition may be different from the amount of light of the light sourceunder the second imaging condition. The image processing unit maygenerate a third image by synthesizing the first image and the secondimage.

According to a nineteenth aspect of the present invention, in the thirdaspect, the observation optical system may include a first opticalsystem, a second optical system, and the switching unit. The firstoptical system may be disposed on an optical front side of the imagingdevice and may form a first optical image of the subject on the imagingdevice. The second optical system may be disposed on an optical frontside of the imaging device and may form a second optical image of thesubject on the imaging device. The switching unit may select either oneof the first optical system and the second optical system and cause onlyeither one of the first optical image and the second optical image to beformed on the imaging device. A visual field of the first optical systemand a visual field of the second optical system may have a commonregion. The switching control unit may switch the optical image formedon the imaging device by controlling the switching unit. The firstoptical image may be formed on the imaging device under the firstimaging condition. The second optical image may be formed on the imagingdevice under the second imaging condition. The image processing unit mayalign a region of the first image corresponding to the common region anda region of the second image corresponding to the common region andgenerate a third image by synthesizing the first image and the secondimage.

According to a twentieth aspect of the present invention, in the thirdaspect, a focus of the observation optical system under the firstimaging condition may be different from a focus of the observationoptical system under the second imaging condition. The control unit maycause the switching unit to set the first imaging condition during aplurality of first frame periods. The control unit may cause theswitching unit to set the second imaging condition during a plurality ofsecond frame periods. Each second frame period of the plurality ofsecond frame periods may be different from each first frame period ofthe plurality of first frame periods. The image processing unit maygenerate a fourth image by executing a noise reduction process on aplurality of first images. The image processing unit may generate afifth image by executing the noise reduction process on a plurality ofsecond images. The image processing unit may generate the third image bysynthesizing the fourth image and the fifth image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view showing an overall configuration of anendoscope device according to a first embodiment of the presentinvention.

FIG. 2 is a block diagram showing a detailed configuration of theendoscope device according to the first embodiment of the presentinvention.

FIG. 3 is a block diagram showing a functional configuration of a CPUaccording to the first embodiment of the present invention.

FIG. 4 is a block diagram showing a configuration of an imaging deviceaccording to the first embodiment of the present invention.

FIG. 5 is a circuit diagram showing a configuration of a pixel accordingto the first embodiment of the present invention.

FIG. 6 is a diagram showing an array of color filters according to thefirst embodiment of the present invention.

FIG. 7 is a timing chart showing an operation of the imaging deviceaccording to the first embodiment of the present invention.

FIG. 8 is a timing chart showing an operation of the imaging deviceaccording to the first embodiment of the present invention.

FIG. 9 is a flowchart showing a procedure of an operation of theendoscope device according to the first embodiment of the presentinvention.

FIG. 10 is a flowchart showing a procedure of the operation of theendoscope device according to the first embodiment of the presentinvention.

FIG. 11 is a timing chart showing an operation of an imaging deviceaccording to a second embodiment of the present invention.

FIG. 12 is a block diagram showing a detailed configuration of anendoscope device according to a third embodiment of the presentinvention.

FIG. 13 is a timing chart showing an operation of an imaging deviceaccording to the third embodiment of the present invention.

FIG. 14 is a flowchart showing a procedure of an operation of theendoscope device according to the third embodiment of the presentinvention.

FIG. 15 is a block diagram showing a detailed configuration of anendoscope device according to a fourth embodiment of the presentinvention.

FIG. 16 is a block diagram showing a configuration of a stripegeneration unit according to the fourth embodiment of the presentinvention.

FIG. 17 is a timing chart showing an operation of an imaging deviceaccording to the fourth embodiment of the present invention.

FIG. 18 is a flowchart showing a procedure of an operation of theendoscope device according to the fourth embodiment of the presentinvention.

FIG. 19 is a block diagram showing a detailed configuration of anendoscope device according to a fifth embodiment of the presentinvention.

FIG. 20 is a block diagram showing a functional configuration of acentral processing unit (CPU) according to the fifth embodiment of thepresent invention.

FIG. 21 is a flowchart showing a procedure of an operation of theendoscope device according to the fifth embodiment of the presentinvention.

FIG. 22 is a flowchart showing a procedure of the operation of theendoscope device according to the fifth embodiment of the presentinvention.

FIG. 23 is a block diagram showing a detailed configuration of anendoscope device according to a sixth embodiment of the presentinvention.

FIG. 24 is a flowchart showing a procedure of an operation of theendoscope device according to the sixth embodiment of the presentinvention.

FIG. 25 is a block diagram showing a detailed configuration of anendoscope device according to a seventh embodiment of the presentinvention.

FIG. 26 is a timing chart showing an operation of an imaging deviceaccording to a reference form of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the drawings.

First Embodiment

FIG. 1 shows an overall configuration of an endoscope device 1 accordingto a first embodiment of the present invention. FIG. 2 shows an internalconfiguration of the endoscope device 1. The endoscope device 1 shown inFIG. 1 has an endoscope 2, a main body unit 3, an operation unit 4, anda display unit 5. The endoscope 2 has an elongated insertion unit 20.

The insertion unit 20 is inserted into a physical object to be observed.An optical adapter 21 can be attached to a tip of the insertion unit 20.The optical adapter 21 has an optical system for taking light from asubject into the tip of the insertion unit 20. For example, theendoscope device 1 can acquire two optical images corresponding to aplurality of different viewpoints by attaching the stereo opticaladapter to the tip of the insertion unit 20. Using the two opticalimages, the endoscope device 1 can measure dimensions of the subject bythe principle of triangulation. The main body unit 3 has a configurationfor controlling the endoscope device 1. The operation unit 4 receives anoperation performed by a user. The display unit 5 (a display) displaysimages acquired by the endoscope device 1, processing menus, and thelike.

FIG. 2 shows a detailed configuration of the endoscope device 1. Theoptical adapter 21 is attached to the tip of the insertion unit 20 shownin FIG. 2. The optical adapter 21 of the first embodiment is a stereooptical adapter for forming a plurality of optical images correspondingto a plurality of viewpoints. The optical adapter 21 has an observationoptical system 60. The optical adapter 21 has a part of an illuminationoptical system 80. The observation optical system 60 has a concave lens23 a, a concave lens 23 b, a convex lens 24 a, a convex lens 24 b, aswitching unit 25, and an image forming optical system 26. Theillumination optical system 80 includes a condenser lens 81, a lightguide 82, a rod lens 83, and a diffusion lens 84. The condenser lens 81is disposed in the main body unit 3. The light guide 82 is disposed inthe main body unit 3 and the insertion unit 20. The rod lens 83 and thediffusion lens 84 are disposed in the optical adapter 21.

The insertion unit 20 has an imaging device 22 (an image sensor). Theimaging device 22 is disposed on the tip of the insertion unit 20. Themain body unit 3 includes an imaging control unit 30, a video processingunit 31, a light source unit 32, an illumination control unit 33, a CPU34, a memory 35, and a switching control unit 36.

The light source unit 32 includes a white light source 37.

The outline of the configuration shown in FIG. 2 will be described. Thewhite light source 37 generates illumination light. The illuminationoptical system 80 radiates the illumination light to the subject. Theobservation optical system 60 forms an optical image of the subject. Theimaging device 22 has a plurality of cells 54 disposed in a matrix andimages the subject. A configuration of the cell 54 which is a pixel ofthe imaging device 22 is shown in FIG. 4. The configuration of the cell54 will be described below. The switching unit 25 (a switching device)performs switching between a plurality of imaging conditions so that theimaging device 22 images the subject. The imaging control unit 30, theillumination control unit 33, and the switching control unit 36 arecontrol units (controllers).

During a first period, the imaging device 22 sequentially reads pixelsignals from at least some of the plurality of cells 54 row by row. Thepixel signals are generated on the basis of the optical image of thesubject. The imaging device 22 generates an image of the subject duringeach frame period of a plurality of frame periods on the basis of atleast some of pixel signals (pixel signals of valid pixels) read from atleast some of the plurality of cells 54. The illumination control unit33 causes the white light source 37 to generate the illumination lightduring a second period. The second period is at least a part of a periodother than the first period. The illumination control unit 33 causes thewhite light source 37 to stop the generation of the illumination lightduring a third period. The third period is all of a period other thanthe second period and includes the first period. The second period andthe third period are alternately iterated. The switching control unit 36causes the switching unit 25 to start switching of the imaging conditionduring the third period and causes the switching of the imagingcondition to be completed during the third period.

A frame is a set of a plurality of pixel signals included in one image.One image (one frame) is generated during one frame period. The imagingdevice 22 generates one image on the basis of pixel signals of oneframe.

Details of the configuration shown in FIG. 2 will be described. Thewhite light source 37 converts power supplied from the illuminationcontrol unit 33 into white light. Thereby, the white light source 37generates illumination light for illuminating the subject. For example,the white light source is a combination of a semiconductorlight-emitting element and a phosphor. The semiconductor light-emittingelement is a light-emitting diode (LED), a laser diode (LD) or the like,and emits blue light. The phosphor converts the blue light, which isexcitation light, into white light. The semiconductor light-emittingelement can perform switching between ON and OFF of light at high speed.The illumination control unit 33 can adjust an exposure time period inthe imaging of the subject by adjusting a turning-on period of thesemiconductor light-emitting element. Light emission efficiency of thesemiconductor light-emitting element is higher than that of anotherlight source such as a halogen lamp. Therefore, power consumption isless than that of other light sources of the same brightness and theendoscope device 1 can be miniaturized.

The illumination control unit 33 supplies electric power to the lightsource unit 32. The illumination control unit 33 controls a timing atwhich the white light source 37 is turned on, a timing at which thewhite light source 37 is turned off, and the amount of light emittedfrom the white light source 37 on the basis of light source controlparameters output from the video processing unit 31. A control mode ofthe white light source 37 includes a continuous turning-on mode and apulse turning-on mode. In the continuous turning-on mode, the amount oflight is controlled by a magnitude of an electric direct currentsupplied to the light source unit 32. In the pulse turning-on mode, theamount of light is controlled by a width and a height of an electriccurrent pulse supplied to the light source unit 32. The white lightsource 37 may be disposed in the optical adapter 21.

The white light emitted from the white light source 37 is concentratedby the condenser lens 81. The white light concentrated by the condenserlens 81 is transmitted to the tip of the insertion unit 20 via thedisposed light guide 82. The light guide 82 is an optical fiber bundleformed by bundling the strands of the optical fiber. The white lightemitted from the light guide 82 is transmitted through the opticaladapter 21 by the rod lens 83. The white light emitted from the rod lens83 is radiated to the subject by the diffusion lens 84.

The observation optical system 60 takes in light reflected on a surfaceof the subject illuminated with the white light. The light taken in bythe observation optical system 60 is incident on the imaging device 22.The observation optical system 60 forms an optical image of the subjectilluminated with the illumination light on the imaging device 22.

The observation optical system 60 includes a first optical system and asecond optical system. The first optical system and the second opticalsystem are disposed on an optical front side (a subject side) of theimaging device 22. The first optical system forms a first optical imageof the subject corresponding to a first viewpoint on the imaging device22. The second optical system forms a second optical image of thesubject corresponding to a second viewpoint different from the firstviewpoint on the imaging device 22.

The concave lens 23 a, the convex lens 24 a, and the image formingoptical system 26 are the first optical system. The first optical systemforms the first optical image based on light from the subject in animaging region S1 of the imaging device 22. An optical path which passesthrough the first optical system is a first optical path L1. The concavelens 23 b, the convex lens 24 b, and the image forming optical system 26are the second optical system. The second optical system forms a secondoptical image based on the light from the subject in the imaging regionS1 of the imaging device 22. The optical path that passes through thesecond optical system is a second optical path L2.

Under the first imaging condition, the switching unit 25 causes lightthat passes through the first optical system to be incident on theimaging device 22 and blocks light that passes through the secondoptical system. Under the second imaging condition, the switching unit25 causes light that passes through the second optical system to beincident on the imaging device 22 and blocks light that passes throughthe first optical system. The switching control unit 36 switches theoptical image formed on the imaging device 22 between the first opticalimage and the second optical image by controlling the switching unit 25.

The switching unit 25 forms only either one of the first optical imageof the subject and the second optical image of the subject in theimaging region S1 of the imaging device 22 by setting either one of thefirst optical path L1 and the second optical path L2 as an imagingoptical path. The first optical path L1 and the second optical path L2are different from each other. The first optical image of the subject isformed by light passing through the first optical path L1. The secondoptical image of the subject is formed by light passing through thesecond optical path L2.

An optical axis on a subject side of the second optical system issubstantially parallel to an optical axis on a subject side of the firstoptical system. The second optical system has parallax with respect tothe first optical system. That is, the first optical system and thesecond optical system are separated in a parallax direction. Theparallax direction is a direction of a straight line that passes throughan optical center (a principal point) of the first optical system and anoptical center (a principal point) of the second optical system. Theparallax direction is substantially orthogonal to the optical axis ofeach optical system. The light incident on the first optical systempasses through the first optical path L1. The light incident on thesecond optical system passes through the second optical path L2different from the first optical path L1. The first optical system formsthe first optical image of the subject and the second optical systemforms the second optical image of the subject.

The switching unit 25 switches the imaging optical path between thefirst optical path L1 and the second optical path L2. The switching unit25 causes only light that passes through either one of the first opticalpath L1 and the second optical path L2 to be transmitted and blockslight that passes through the other. For example, the switching unit 25includes a shutter (a shielding plate) inserted into only either one ofthe first optical path L1 and the second optical path L2. When theswitching unit 25 causes the light of the first optical path L1 to betransmitted, the shutter is inserted into the second optical path L2 andthe light of the second optical path L2 is blocked. When the switchingunit 25 causes the light of the second optical path L2 to betransmitted, the shutter is inserted into the first optical path L1 andthe light of the first optical path L1 is blocked. The operation of theswitching unit 25 is controlled by a control signal from the switchingcontrol unit 36. The switching unit 25 may be a liquid crystal shutterincluding a polarizing plate and a liquid crystal cell. The switchingunit 25 is not limited to the above-described configuration.

The image forming optical system 26 forms a subject image based oneither one of the light passing through the first optical path L1 andthe light passing through the second optical path L2 in the imagingregion S1 of the imaging device 22. The subject image formed in theimaging region S1 of the imaging device 22 is based on light passingthrough only an imaging optical path between the first optical path L1and the second optical path L2. The imaging optical path is either oneof the first optical path L1 and the second optical path L2.

The optical adapter 21 and the insertion unit 20 may be integrated. Thatis, the configuration inside the optical adapter 21 may be disposed onthe tip of the insertion unit 20.

The first optical image of the subject is formed on the basis of thelight passing through the first optical path L1. The second opticalimage of the subject is formed on the basis of light passing through thesecond optical path L2. The first optical image and the second opticalimage are incident in the imaging region S1 of the imaging device 22.The imaging device 22 captures the first optical image and the secondoptical image. The imaging device 22 captures the first optical imageformed by the first optical system at a first imaging timing. Theimaging device 22 captures the second optical image formed by the secondoptical system at a second imaging timing. The first imaging timing andthe second imaging timing are different from each other.

The imaging device 22 consecutively scans a plurality of rows in thearray of the plurality of cells 54 one by one during each frame periodof a plurality of frame periods. The imaging device 22 reads pixelsignals from the cells 54 in a plurality of rows by consecutivelyscanning the plurality of rows one by one. The imaging device 22generates a first image and a second image. The first image is formed onthe basis of the first optical image formed in the imaging region S1.The second image is formed on the basis of the second optical imageformed in the imaging region S1. The first image and the second imageare image data including pixel values based on the pixel signals readfrom the cells 54 of the plurality of rows. The imaging device 22outputs the first image and the second image to the video processingunit 31. The operation of the imaging device 22 is controlled by acontrol signal from the imaging control unit 30.

The plurality of imaging conditions include a first imaging conditionand a second imaging condition. The first imaging condition and thesecond imaging condition are different from each other. Under the firstimaging condition, the first optical path L1 is set as the imagingoptical path. The imaging device 22 generates a first image of thesubject by imaging the subject under the first imaging condition. Underthe second imaging condition, the second optical path L2 is set as theimaging optical path. The imaging device 22 generates a second image ofthe subject by imaging the subject under the second imaging condition.

For example, a line exposure type of CMOS imager is used for the imagingdevice 22. By adopting a CMOS imager, the configuration of the endoscopedevice 1 can be simplified and the power consumption of the endoscopedevice 1 can be reduced.

A signal line 90 is disposed inside the insertion unit 20 and inside themain body unit 3. The signal line 90 is a composite coaxial line formedby bundling a plurality of coaxial cables. A tip side of the signal line90 is connected to the imaging device 22, and a part of the coaxialcable on a base end side of the signal line 90 is connected to theimaging control unit 30. The imaging control unit 30 supplies electricpower for driving to the imaging device 22 via the signal line 90. Also,the imaging control unit 30 outputs an imaging parameter received fromthe video processing unit 31 to the imaging device 22. Thereby, theimaging control unit 30 controls the imaging device 22.

The remaining coaxial cable on the base end side of the signal line 90is connected to the video processing unit 31. The image generated by theimaging device 22 is transmitted to the video processing unit 31. Thevideo processing unit 31 executes various types of video processing onthe image output from the imaging device 22. For example, the videoprocessing to be executed by the video processing unit 31 is at leastone of demosaicing, digital gain adjustment, noise reduction, whitebalance adjustment, contour correction, and gamma correction. The videoprocessing unit 31 synthesizes an image on which the video processinghas been performed and graphic data generated by the CPU 34. Thereby,the video processing unit 31 generates a video signal for display. Thevideo processing unit 31 outputs the generated video signal to thedisplay unit 5.

Further, the video processing unit 31 generates control parameters forperforming imaging with appropriate brightness. The video processingunit 31 generates imaging control parameters and illumination controlparameters on the basis of an input image or an image on which the videoprocessing has been performed. The imaging control parameters areparameters such as a line reading cycle, a frame rate, and an analoggain of the imaging device 22. The illumination control parameters areparameters such as an ON timing of the illumination light, an OFF timingof the illumination light, and a turning-on intensity. The videoprocessing unit 31 outputs the imaging control parameters to the imagingcontrol unit 30. The imaging control unit 30 controls the imaging device22 on the basis of the imaging control parameters. The video processingunit 31 outputs the illumination control parameters to the illuminationcontrol unit 33. The illumination control unit 33 controls the whitelight source 37 on the basis of the illumination control parameters.

At least two of the imaging control unit 30, the illumination controlunit 33, and the switching control unit 36 may be integrated. Theimaging control unit 30, the illumination control unit 33, and theswitching control unit 36 may include at least one of a processor and alogic circuit. For example, the processor is at least one of a CPU, adigital signal processor (DSP), and a graphics processing unit (GPU).For example, the logic circuit is at least one of an applicationspecific integrated circuit (ASIC) and a field-programmable gate array(FPGA). The imaging control unit 30, the illumination control unit 33,and the switching control unit 36 can include one or more processors.The imaging control unit 30, the illumination control unit 33, and theswitching control unit 36 can include one or more logic circuits.

A computer of the endoscope device 1 may read a program and execute theread program. The program includes commands defining operations of theimaging control unit 30, the illumination control unit 33, and theswitching control unit 36. That is, the functions of the imaging controlunit 30, the illumination control unit 33, and the switching controlunit 36 may be implemented by software. For example, this program may beprovided by a “computer-readable recording medium” such as a flashmemory. The above-described program may be transmitted from a computerhaving a storage device or the like in which the program is stored tothe endoscope device 1 via a transmission medium or transmission wavesin the transmission medium. The “transmission medium” for transmittingthe program refers to a medium having an information transmissionfunction, for example, a network (a communication network) such as theInternet or a communication circuit (a communication line) such as atelephone circuit. Also, the above-described program may be a programfor implementing some of the above-described functions. Further, theabove-described program may be a program capable of implementing theabove-described function in combination with a program already recordedon the computer, i.e., a so-called differential file (differentialprogram). A combination of a program already recorded in the computerand a differential program may implement the above-described functions.

The CPU 34 controls each unit in the endoscope device 1. Further, theCPU 34 monitors the state of the operation unit 4. Thereby, the CPU 34detects operations related to measurement and the like. The CPU 34 maybe a DSP or a GPU. The CPU 34 may be an ASIC or an FPGA.

The memory 35 stores the image processed by the video processing unit31. The memory 35 may store the image output from the imaging device 22.The memory 35 may be detachable from the endoscope device 1. The memory35 is configured as a volatile or nonvolatile memory. For example, thememory 35 may be any one of a random access memory (RAM), a dynamicrandom access memory (DRAM), a static random access memory (SRAM), anerasable programmable read only memory (EPROM), an electrically erasableprogrammable read only memory (EEPROM), and a flash memory.Alternatively, the memory 35 may be a combination of at least two of theabove-described memories. The endoscope device 1 may have a hard diskdrive for storing images.

The operation unit 4 is a user interface that accepts instructions fromthe user. The user inputs instructions necessary for controlling varioustypes of operations of the entire endoscope device 1 by operating theoperation unit 4. The operation unit 4 outputs a signal indicating aninstruction received from the user to the CPU 34. For example, theoperation unit 4 is at least one of a button, a switch, a key, a mouse,a joystick, a touch pad, a track ball, and a touch panel.

The display unit 5 displays an image of the subject on the basis ofvideo signals output from the video processing unit 31. Also, thedisplay unit 5 displays operation control details, measurement results,and the like. For example, the operation control details are displayedas a menu. For example, the display unit 5 is at least one of a liquidcrystal display and an organic electro luminescence (EL) display. Thedisplay unit 5 may be a touch panel display. In this case, the operationunit 4 and the display unit 5 are integrated.

FIG. 3 shows a functional configuration of the CPU 34. The functions ofthe CPU 34 include a control unit 340, a display processing unit 341, animage processing unit 342, a visualization processing unit 343, and arecording unit 344. At least one of blocks shown in FIG. 3 may include acircuit different from the CPU 34.

The control unit 340 controls a process to be executed by each unit.When the user operates the operation unit 4, the control unit 340accepts an operation performed by the user. The display processing unit341 generates graphic data for displaying a menu and the like. Thegraphic data generated by the display processing unit 341 is output tothe video processing unit 31. Also, the display processing unit 341controls the state of an image to be displayed on the display unit 5 bycontrolling the video processing unit 31.

The image processing unit 342 (an image processor) executes imageprocessing on the basis of the image of the subject. The imageprocessing unit 342 includes a suitability determination unit 345, anoise reduction unit 346, and a measurement processing unit 347.

The image processing unit 342 executes image processing on a pluralityof images generated during a plurality of frame periods. The imageprocessing unit 342 processes a plurality of images.

The switching control unit 36 causes the switching unit 25 to set thefirst imaging condition during a plurality of first frame periods. Theswitching control unit 36 causes the switching unit 25 to set the secondimaging condition during a plurality of second frame periods. Eachsecond frame period of the plurality of second frame periods isdifferent from each first frame period of the plurality of first frameperiods.

The imaging device 22 generates a plurality of at least one of firstimages of the subject and second images of the subject. The suitabilitydetermination unit 345 calculates at least one of a value indicatingwhether or not the plurality of first images are suitable for imageprocessing and a value indicating whether or not the plurality of secondimages are suitable for the image processing.

The imaging device 22 generates the plurality of first images by imagingthe subject under the first imaging condition. The imaging device 22generates the plurality of second images by imaging the subject underthe second imaging condition. The noise reduction unit 346 generates athird image by performing a noise reduction process on the basis of theinput of the plurality of first images. The noise reduction unit 346generates a fourth image by performing a noise reduction process on thebasis of the input of the plurality of second images.

The image processing unit 342 performs a process different from thenoise reduction process on the third image and the fourth image.Specifically, the measurement processing unit 347 calculates 3Dcoordinates of a plurality of points on a surface of the subject using apassive stereo method on the basis of the third image and the fourthimage.

The measurement processing unit 347 measures at least one of a shape ofthe subject, dimensions of the subject, and a distance to the subject (asubject distance). For example, the shape of the subject is measured asa 3D point cloud or a mesh polygon.

The 3D point cloud is a set of 3D coordinates of a plurality of pointson the surface of the subject. The mesh polygon is a set of triangleshaving each point included in the 3D point cloud as its vertex. Thedimensions of the subject are a distance between any two points on thesubject, an area of a region including three or more points on thesubject, and the like. The subject distance is a distance from the tipof the insertion unit 20 where the imaging device 22 is disposed to thesubject. Specifically, the subject distance is a distance from theimaging device 22 to the subject. The subject distance may be a distancefrom a principal point of the first optical system or a principal pointof the second optical system to the subject. The subject distance may bea distance from a surface of the subject side of the lens to thesubject. The measurement processing unit 347 performs stereo measurementby triangulation using disparities of the two images. Specifically, themeasurement processing unit 347 detects a point corresponding to ameasurement point set in the first image or the third image from thesecond image or the fourth image. This process is referred to as atemplate matching process. The measurement processing unit 347calculates 3D coordinates of a point on the subject corresponding to themeasurement point coordinates on the basis of the coordinates of thedetected point (corresponding point coordinates) and the coordinates ofthe measurement point (measurement point coordinates). The measurementpoint is associated with an address of the cell 54 in the imaging regionS1 of the imaging device 22. The measurement processing unit 347 maycalculate 3D coordinates of only one point on the surface of thesubject.

The result of the image processing executed by the image processing unit342 is output to the visualization processing unit 343. Thevisualization processing unit 343 is a data generation unit (a datagenerator). The visualization processing unit 343 generates graphic datathat is obtained by visualizing the image processing result. Thereby,the visualization processing unit 343 generates graphic data that isobtained by representing the result of reconstructing a 3D shape of thesubject as an image.

In the first embodiment, the illumination light is white light. Theimaging device 22 has a red color filter, a green color filter, and ablue color filter. Each color filter is shown in FIG. 6. Each colorfilter will be described below. The imaging device 22 generates a colorimage as an image of a subject. The color image includes a plurality ofpixels. Each pixel has information indicating each of brightness of red,brightness of green, and brightness of blue as a pixel value. The imageprocessing unit 342 generates 3D shape data (a color 3D point cloud or amesh polygon with color texture) in which 3D coordinates (a 3D pointcloud) of a plurality of points on the surface of the subject areassociated with pixel values corresponding to the plurality of points.The visualization processing unit 343 generates graphic data that isobtained by visualizing the 3D shape data. The graphic data correspondsto an image that is obtained by disposing the 3D shape data within avirtual space and photographing the 3D shape data with a virtual cameradisposed within the virtual space. In the 3D shape data generated by theimage processing unit 342, 3D coordinates of each point included in aplurality of points on the surface of the subject are associated with apixel value of each point included in the plurality of points. At leastone of 3D shape data disposed in the virtual space by the visualizationprocessing unit 343, a position of the camera, an orientation of thecamera, and an angle of view of the camera is changed by the useroperating the operation unit 4.

The graphic data generated by the visualization processing unit 343 isoutput to the video processing unit 31. The recording unit 344 recordsthe image of the subject in the memory 35.

The operation unit 4 and the display unit 5 are optional.

FIG. 4 shows a configuration of the imaging device 22. The imagingdevice 22 shown in FIG. 4 includes a pixel unit 50, a vertical scanningunit 51, a signal processing unit 52, and a horizontal scanning unit 53.

The pixel unit 50 includes a plurality of cells 54 disposed in a matrix.The plurality of cells 54 are disposed in the imaging region S1 of theimaging device 22. Each of the number of rows and the number of columnsin the array of the plurality of cells 54 is two or more. The number ofrows and the number of columns may not be the same. Each cell 54 of theplurality of cells 54 generates a pixel signal corresponding to theamount of light incident on the cell 54. Each cell 54 of the pluralityof cells 54 is connected to the vertical signal line 56. A plurality ofvertical signal lines 56 are disposed. Each cell 54 of the plurality ofvertical signal lines 56 is disposed for one column in the array of theplurality of cells 54. Each cell 54 of the plurality of cells 54 outputsthe generated pixel signal to the vertical signal line 56.

Each cell 54 of the plurality of cells 54 is connected to the controlsignal line 57. A plurality of control signal lines 57 are disposed.Each control signal line 57 of the plurality of control signal lines 57is disposed for one row in the array of the plurality of cells 54. Eachcontrol signal line 57 of the plurality of control signal lines 57 isconnected to the vertical scanning unit 51. Control signals forcontrolling operations of the plurality of cells 54 are output from thevertical scanning unit 51 to the control signal line 57. The pluralityof control signal lines 57 are disposed for cells 54 of one row. In FIG.4, one control signal line 57 is shown for the cells 54 of one row andthe other control signal lines 57 are omitted. Details of the controlsignals will be described below.

The operations of the plurality of cells 54 are controlled on the basisof the control signals output to the control signal lines 57. Thecontrol signals corresponding to the cells 54 of one row are supplied incommon to all the cells 54 in the row. Thus, the same operation timingis set for two or more cells 54 disposed in the same row. That is, thetwo or more cells 54 disposed in the same row operate simultaneously.Details of the configuration of the cell 54 will be described below.

A control signal generated by the imaging control unit 30 is transmittedto the imaging device 22. The vertical scanning unit 51 generates thecontrol signals for controlling the operations of the plurality of cells54 on the basis of the control signal from the imaging control unit 30.The vertical scanning unit 51 generates control signals corresponding toeach row of a plurality of rows in the array of the plurality of cells54. The vertical scanning unit 51 outputs the generated control signalsto the control signal lines 57.

The signal processing unit 52 includes a plurality of signal processingcircuits 55. The signal processing circuit 55 is disposed for eachcolumn in the array of the plurality of cells 54. The signal processingcircuit 55 is connected to the vertical signal line 56. The signalprocessing circuit 55 performs signal processing on a pixel signaloutput from the cell 54 to the vertical signal line 56. The signalprocessing to be performed by the signal processing circuit 55 includescorrelated double sampling (CDS), analog gain control (AGC), and thelike.

The pixel signal processed by the signal processing circuit 55 is inputto the horizontal scanning unit 53. The horizontal scanning unit 53sequentially selects columns in the array of the plurality of cells 54.The pixel signal corresponding to the column selected by the horizontalscanning unit 53 is output from the output terminal 58.

The imaging device 22 includes the plurality of cells 54 disposed in thematrix. During each frame period of the plurality of frame periods, theimaging device 22 generates pixel signals of the cells 54 based on anoptical image of the subject. During each frame period of the pluralityof frame periods, the imaging device 22 generates an image of thesubject using the pixel signals.

FIG. 5 shows a configuration of the cell 54. The cell 54 shown in FIG. 5includes a photoelectric conversion unit 70, a charge transfer unit 71,a charge holding unit 72, a capacitor reset unit 73, an amplificationunit 74, and an output unit 75. The photoelectric conversion unit 70 isa photodiode. The charge holding unit 72 is a capacitor. The chargetransfer unit 71, the capacitor reset unit 73, the amplification unit74, and the output unit 75 are transistors.

The photoelectric conversion unit 70 generates and stores electriccharge according to the amount of light incident on the cell 54. Thecharge transfer unit 71 transfers the electric charge generated andstored by the photoelectric conversion unit 70 to the charge holdingunit 72. The charge holding unit 72 holds the electric chargetransferred from the photoelectric conversion unit 70. The capacitorreset unit 73 resets the electric charge of the charge holding unit 72on the basis of a power-supply voltage VDD. The capacitor reset unit 73is turned on and therefore the capacitor reset unit 73 resets theelectric charge of the charge holding unit 72. The amplification unit 74amplifies a signal based on the electric charge held in the chargeholding unit 72. The output unit 75 outputs the signal amplified by theamplification unit 74 as a pixel signal to the vertical signal line 56.

The operation of the charge transfer unit 71 is controlled by a controlsignal ϕTX. The operation of the capacitor reset unit 73 is controlledby a control signal ϕRST. The operation of the output unit 75 iscontrolled by a control signal ϕSEL. The control signal ϕTX, the controlsignal ϕRST, and the control signal ϕSEL are supplied from the verticalscanning unit 51 via the control signal line 57.

The operation of the cell 54 includes a capacitor reset operation, acharge transfer operation, and a signal reading operation. The capacitorreset operation corresponds to the operation of the capacitor reset unit73. The charge transfer operation corresponds to the operation of thecharge transfer unit 71. The signal reading operation corresponds to theoperation of the output unit 75. A period from a storage start timing toa transfer timing is a period (an exposure period) during which exposurecan be performed in the cell 54. The storage start timing is a timing atwhich the photoelectric conversion unit 70 starts generation of electriccharge based on the light incident on the cell 54 and storage of theelectric charge. The transfer timing is a timing at which the chargetransfer unit 71 transfers the electric charge stored in thephotoelectric conversion unit 70 to the charge holding unit 72. The cell54 stores electric charge in accordance with the amount of lightincident on the cell 54 during the exposure period of the cell 54. Inthe following description, a state in which the cell 54 has been resetrepresents a state of the cell 54 at a timing when the exposure periodhas ended and the charge transfer unit 71 has transferred the electriccharge stored in the photoelectric conversion unit 70 to the chargeholding unit 72.

The imaging device 22 reads the pixel signal from the cell 54 byoutputting the pixel signal from the output unit 75. The imaging device22 acquires an image by reading the pixel signal from the cell 54. Thereading of pixel signals is equivalent to the acquisition of an image.

As shown in FIG. 6, the imaging device 22 includes color filters CFdisposed in each cell 54. FIG. 6 shows an array of the color filters CF.The color filters CF include an R filter CFr, a G filter CFgr, a Gfilter CFgb, and a B filter CFb.

The R filter CFr is a red color filter. The R filter CFr causes redlight to be transmitted and blocks other light. The G filter CFgr andthe G filter CFgb are green color filters. The G filter CFgr and the Gfilter CFgb cause green light to be transmitted and block other light.The B filter CFb is a blue color filter. The B filter CFb causes bluelight to be transmitted and blocks other light.

The array of the color filters CF shown in FIG. 6 is a Bayer array. Inthe Bayer array, a basic array is regularly and periodically disposed inthe row direction and the column direction. The basic array includes oneR filter CFr, two G filters CFgr and CFgb, and one B filter CFb.

The imaging device 22 generates an R signal based on the red light, a Gsignal based on the green light, and a B signal based on the blue light.The imaging device 22 outputs a color image including pixel values basedon the R signal, the G signal, and the B signal.

FIG. 7 shows relationships between an operation of the imaging device 22in a measurement mode, a state of illumination, and a state of animaging condition. The operation of the imaging device 22 will bedescribed with reference to FIG. 7. In the example shown in FIG. 7, theimaging region S1 of the imaging device 22 has eight rows. The number ofrows of the imaging region S1 is not limited to eight.

A timing chart TC10 shows the operation of the imaging device 22. In thetiming chart TC10, the horizontal direction represents time and thevertical direction represents a row position of the cell 54. The top rowis a first row and the bottom row is an eighth row.

In the timing chart TC10, the state of illumination, i.e., the state ofthe white light source 37, is shown. The state of the white light source37 is either one of ON and OFF. In the timing chart TC10, the imagingconditions, i.e., imaging optical paths, are shown. The imaging opticalpath is either one of the first optical path L1 and the second opticalpath L2. In the timing chart TC10, the operation of the switching unit25 is shown. The operation of the switching unit 25 is either one ofswitching from the first optical path L1 to the second optical path L2and switching from the second optical path L2 to the first optical pathL1.

In the timing chart TC10, a frame rate is 60 fps. A length of each frameperiod is 1/60 sec. Each frame period includes an exposure period ofcells 54 for one row and a reading period of cells 54 for one row.During a reading period of a certain row, the pixel signals of the cells54 included in the row are read. The reading of the pixel signalsincludes the charge transfer and the signal reading. In the timing chartTC10, a frame period based on a start timing of the exposure period inthe cells 54 of the first row is shown. Frame periods of the second toeighth rows are started with a delay of a predetermined time period fromthe frame period of a row immediately before each row. The pixel signalstored in the cell 54 during the exposure period of the frame period nis read from the cell 54 during the reading period of the frame periodn. The exposure period of the frame period n is the exposure period ofeach of the first to eighth rows of the frame period n. The readingperiod of the frame period n is a period from the start of the readingperiod of the first row of the frame period n to the end of the readingperiod of the eighth row of the frame period n.

A broken line L10 in FIG. 7 indicates a start timing of the frame periodof each row. Although the broken line L10 is omitted in timing chartsother than the timing chart TC10, the meaning of the broken line L10 iscommon in the present specification. A symbol M10 in FIG. 7 indicates areading period. A symbol M11 in FIG. 7 indicates an exposure period.Although the symbols M10 and M11 are omitted in the timing charts otherthan the timing chart TC10, the meanings of the symbols M10 and M11 arecommon in the present specification.

At a start timing of an exposure period of a frame period i, the cells54 of the first row are reset. Thereby, the exposure period of the cells54 of the first row is started. During the exposure period, signalsbased on the light incident on the cells 54 are stored in the cells 54.After the exposure period of the cells 54 of the first row is started,the exposure period of the cells 54 of the second row is started.Likewise, the exposure periods of the cells 54 of the third to eighthrows are sequentially started.

The vertical scanning unit 51 sequentially generates the control signalsof the rows and sequentially outputs the generated control signals tothe cells 54 of the rows. The imaging device 22 consecutivelysequentially resets the cells 54 of the plurality of rows row by row onthe basis of the control signals sequentially output from the verticalscanning unit 51. According to this resetting, the imaging device 22sequentially starts the exposure periods of the cells 54 of a pluralityof rows.

During a frame period (i−1) immediately before the frame period i, theimaging optical path of the acquired image is the second optical pathL2. When the exposure period of the cells 54 of the first row during theframe period i has been started, switching of the imaging optical pathis started. The switching control unit 36 outputs a control signal forswitching the imaging optical path to the switching unit 25. Thereby,the switching control unit 36 causes the switching unit 25 to switch theimaging optical path. The switching unit 25 starts switching from thesecond optical path L2 to the first optical path L1 on the basis of thecontrol signal from the imaging control unit 30. It is only necessaryfor switching from the second optical path L2 to the first optical pathL1 to be started simultaneously with or after the start of the readingof the pixel signals of the cells 54 of the first row during the frameperiod (i−1) (the start of the reading period of the frame period(i−1)). Also, a timing at which the control signal is output from theimaging control unit 30 may be earlier than that of the start of thereading of the pixel signals of the cells 54 of the first row by acertain time period. The certain time is the same as or shorter than atime difference from an output timing of the control signal to a starttiming of switching of the optical path. Thereby, the switching of theoptical path is started simultaneously with or after the start of thereading of the pixel signals of the cells 54 of the first row.

When the exposure period of the cells 54 of the third row during theframe period i has been started, the switching unit 25 completes theswitching from the second optical path L2 to the first optical path L1.The imaging optical path of the image acquired during the frame period iis the first optical path L1. It is only necessary for switching fromthe second optical path L2 to the first optical path L1 to be completedsimultaneously with or before the start of the exposure period of thecells 54 of the eighth row during the frame period i (the end of thereading period of the frame period (i−1)).

When the exposure period of the cells 54 of the first row during theframe period i has been started, the white light source 37 is alreadyturned off. When the exposure period of the cells 54 of the eighth rowduring the frame period i has been started, the turning on of the whitelight source 37 is started. The illumination control unit 33 causes thewhite light source 37 to be turned on by outputting a turning-on controlsignal to the white light source 37. The white light source 37 startsthe turning on thereof on the basis of a control signal from theillumination control unit 33. The turning on of the white light source37 may be started after the start of the exposure period of the cells 54of the eighth row during the frame period i (the end of the readingperiod of the frame period (i−1)).

When the exposure period of the cells 54 of the first row during theframe period i has ended, the reading period of the cells 54 of thefirst row during the frame period i is started. The cells 54 of thefirst row output pixel signals to the vertical signal lines 56. When apredetermined time period has elapsed from a timing at which the readingperiod of the cells 54 of the first row has been started, the readingperiod of the cells 54 of the first row ends. At this moment, the cells54 of the first row are reset and the exposure period of the cells 54 ofthe first row during the frame period (i+1) is started.

When the reading period of the cells 54 of the first row during theframe period i has been started, the turning-on of the white lightsource 37 ends. The illumination control unit 33 causes the white lightsource 37 to be turned off by outputting a turning-off control signal tothe white light source 37. The white light source 37 is turned off onthe basis of the control signal from the illumination control unit 33.The end of the turning-on of the white light source 37 may be earlierthan the start of the reading period of the cells 54 of the first rowduring the frame period i (the start of the reading period of the frameperiod i).

In the timing chart TC10 shown in FIG. 7, the illumination control unit33 causes the white light source 37 to be intermittently turned on. Thewhite light source 37 is intermittently turned on. A period during whichthe white light source 37 is continuously turned on overlaps theexposure periods of the cells 54 of the plurality of rows. During theperiod in which the white light source 37 is continuously turned on, thecells 54 of all rows are simultaneously exposed. A period (an imagingperiod) during which light from the subject is incident on the cells 54in accordance with the turning-on of the white light source 37 is commonin all the cells 54 of all the rows. That is, global exposures(simultaneous exposures) of the cells 54 of the plurality of rows areperformed on the basis of control of a turning-on timing of the whitelight source 37 and a turning-off timing of the white light source 37. Aperiod during which the white light source 37 is continuously turned ondoes not include the reading period of the cell 54.

When the reading period of the cells 54 of the first row during theframe period i has ended, the reading period of the cells 54 of thesecond row during the frame period i is started. The cells 54 of thesecond row output pixel signals to the vertical signal lines 56. When apredetermined time period has elapsed from a timing when the readingperiod of the cells 54 of the second row has been started, the readingperiod of the cells 54 of the second row ends. At this moment, the cells54 of the second row are reset and the exposure period of the cells 54of the second row during the frame period (i+1) is started. Likewise,the reading periods of the cells 54 of the third to eighth rows aresequentially started, and the pixel signals of the cells 54 of the thirdrow to the eighth row are sequentially read. The cells 54 of the thirdto eighth rows are sequentially reset and the exposure periods of thecells 54 of the third to eighth rows during the frame period (i+1) aresequentially started.

The vertical scanning unit 51 sequentially generates control signals ofthe rows and sequentially outputs the generated control signals to thecells 54 of the rows. The imaging device 22 consecutively scans thecells 54 of the plurality of rows row by row on the basis of the controlsignals sequentially output from the vertical scanning unit 51.According to this scanning, the imaging device 22 sequentially readspixel signals from the cells 54 of the plurality of rows.

A method of driving the imaging device 22 is a rolling shutter method.In the rolling shutter method, the rows to be read are changed one byone during the reading period of one frame period and the pixel signalsare consecutively read from the cells 54 of the rows. In the rollingshutter method, exposure periods are sequentially started row by row andpixel signals are sequentially read row by row. The cells 54 of the rowin which pixel signal reading is completed are reset and the exposureperiod of the next frame period is started.

When the reading period of the cells 54 of the first row during theframe period i has ended, switching of the imaging optical path isstarted. The switching control unit 36 outputs a control signal forswitching of the imaging optical path to the switching unit 25. Thereby,the switching control unit 36 causes the switching unit 25 to switch theimaging optical path. The switching unit 25 starts switching from thefirst optical path L1 to the second optical path L2 on the basis of acontrol signal from the imaging control unit 30. It is only necessaryfor the switching from the first optical path L1 to the second opticalpath L2 to be started simultaneously with or after the start of thereading of the cells 54 of the first row during the frame period i (thestart of the reading period of the frame period i).

When the reading period of the cells 54 of the third row during theframe period i has ended, the switching unit 25 completes switching fromthe first optical path L1 to the second optical path L2. The imagingoptical path of the image acquired during the frame period (i+1) is thesecond optical path L2. It is only necessary for switching from thefirst optical path L1 to the second optical path L2 to be completedsimultaneously with or before the start of the exposure period of thecells 54 of the eighth row during the frame period (i+1) (the end of thereading period of the frame period i).

When the reading period of the cells 54 of the eighth row during theframe period i has ended, the turning-on of the white light source 37 isstarted. The illumination control unit 33 causes the white light source37 to be turned on by outputting a turning-on control signal to thewhite light source 37. The white light source 37 starts turning-on onthe basis of the control signal from the illumination control unit 33.The turning-on of the white light source 37 may be started after the endof the reading period of the cells 54 of the eighth row during the frameperiod i (the end of the reading period of the frame period i).

During each frame period before the frame period i, the imaging device22 executes the same operation as the operation during the frame periodi. During each frame period before the frame period i, the white lightsource 37 is intermittently turned on. During each frame period beforethe frame period i, the switching unit 25 switches the imaging opticalpath to the first optical path L1 or the second optical path L2.

When the exposure period of the cells 54 of the first row during theframe period (i+1) has ended, the reading period of the cells 54 of thefirst row during the frame period (i+1) is started. As in the operationduring the frame period i, the reading periods of the cells 54 of thefirst to eighth rows are sequentially started and the pixel signals ofthe cells 54 of the first to eighth rows are sequentially read. Thecells 54 of the first to eighth rows are sequentially reset and theexposure periods of the cells 54 of the first to eighth rows during theframe period (i+2) are sequentially started.

When the reading period of the cells 54 of the first row during theframe period (i+1) has been started, the turning-on of the white lightsource 37 ends. The illumination control unit 33 causes the white lightsource 37 to be turned off by outputting a turning-off control signal tothe white light source 37. The white light source 37 is turned off onthe basis of a control signal from the illumination control unit 33. Theturning-on of the white light source 37 may be ended before the start ofthe reading period of the cells 54 of the first row during the frameperiod (i+1) (the start of the reading period of the frame period(i+1)).

When the reading period of the cells 54 of the first row during theframe period (i+1) has ended, the switching of the imaging optical pathis started. The switching control unit 36 outputs a control signal forswitching the imaging optical path to the switching unit 25. Thereby,the switching control unit 36 causes the switching unit 25 to switch theimaging optical path. The switching unit 25 starts switching from thesecond optical path L2 to the first optical path L1 on the basis of thecontrol signal from the imaging control unit 30. It is only necessaryfor the switching from the second optical path L2 to the first opticalpath L1 to be started simultaneously with or after the start of readingof the cells 54 of the first row during the frame period (i+1) (thestart of the reading period during the frame period (i+1)).

When the reading period of the cells 54 of the third row during theframe period (i+1) has ended, the switching unit 25 completes switchingfrom the second optical path L2 to the first optical path L1. Theimaging optical path of the image acquired during the frame period (i+2)is the first optical path L1. It is only necessary for switching fromthe second optical path L2 to the first optical path L1 to be completedsimultaneously with or before the start of the exposure period of thecells 54 of the eighth row during the frame period (i+2) (the end of thereading period of the frame period

During the frame period (i+2) and the frame period (i+3), the imagingdevice 22 executes the same operation as the operation during the frameperiod i.

The white light source 37 is turned on during the exposure period ineach of the frame period (i+2) and the frame period (i+3). The whitelight source 37 is iteratively turned on and off. The white light source37 is turned on during each frame period and is turned off during eachframe period.

The switching unit 25 switches the imaging optical path during thereading period in each of the frame period (i+2) and the frame period(i+3). The switching unit 25 iterates switching from the first opticalpath L1 to the second optical path L2 and switching from the secondoptical path L2 to the first optical path L1. The switching unit 25switches the imaging optical path during each frame period.

While the endoscope device 1 is operating in the measurement mode, theendoscope device 1 consecutively or intermittently iterates theoperation during the frame periods i to (i+3).

In the timing chart TC10 shown in FIG. 7, pixel signals are read fromthe cells 54 of the first to eighth rows. That is, the pixel signals areread from all of the plurality of cells 54. The pixel signals may beread from the cells 54 of only some rows. For example, pixel signals maybe read from cells 54 of a row other than a row including only opticalblack pixels or pixel signals may be read from cells 54 of only a validrow that is a row including valid pixels. When pixel signals are readfrom the cells 54 of only some rows, a period during which the whitelight source 37 is continuously turned on may overlap a period duringwhich the cells 54 of only some rows are simultaneously exposed. Thenumbers of the first to eighth rows indicate an order of reading. Thenumbers may be different from those of a physical order on an imagingplane. For example, the cells 54 may be physically disposed indescending order from the eighth row. Alternatively, the cells 54 may bedisposed in the order of the first row, the fifth row, the second row,the sixth row, the third row, the seventh row, the fourth row, and theeighth row in a predetermined direction.

During a first period T1, the imaging device 22 sequentially reads pixelsignals from at least some of the plurality of cells 54 row by row.During the first period T1, the imaging device 22 sequentially readspixel signals from the cells 54 of simultaneous exposure rows row byrow. The simultaneous exposure rows are at least some of the pluralityof rows in the array of the plurality of cells 54. The simultaneousexposure rows include cells 54 to be simultaneously exposed by turningon the white light source 37. In the timing chart TC10 shown in FIG. 7,all rows are simultaneous exposure rows and the first period T1 of theframe period n is a period from the start of the reading of the pixelsignals in the cells 54 of the first row of the frame period n to thecompletion of the reading of the pixel signals in the cells 54 of theeighth row (the reading period of the frame period n).

In the timing chart TC10 shown in FIG. 7, the white light source 37 isturned on simultaneously with the completion of reading of pixel signalsduring one frame period. That is, the white light source 37 is turned onsimultaneously with the completion of the first period T1. The whitelight source 37 is turned off simultaneously with the start of thereading of pixel signals during one frame period. That is, the whitelight source 37 is turned off simultaneously with the start of the firstperiod T1.

The illumination control unit 33 causes the white light source 37 togenerate the illumination light during the second period T2. The whitelight source 37 is continuously turned on during the second period T2.The second period T2 is at least a part of a period other than the firstperiod T1. In the timing chart TC10 shown in FIG. 7, the second periodT2 is all of the period other than the first period T1.

The illumination control unit 33 causes the white light source 37 tostop the generation of the illumination light during the third periodT3. The white light source 37 is continuously turned off during thethird period T3. The third period T3 is all of a period other than thesecond period T2. The third period T3 is a period including all of thefirst period T1. In the timing chart TC10 shown in FIG. 7, the thirdperiod T3 is the same as the first period T1.

The second period T2 overlaps the exposure periods of a plurality ofrows in the array of the plurality of cells 54. When the reading periodof the row finally scanned by the imaging device 22 during one frameperiod has been completed, the second period T2 is started. When thereading period of the row first scanned by the imaging device 22 duringone frame period has been started, the second period T2 is completed.

The switching control unit 36 causes the switching unit 25 to startswitching of the imaging optical path during the third period T3 andcomplete the switching of the imaging optical path during the thirdperiod T3. That is, before the next second period T2 is started, theswitching control unit 36 causes the switching unit 25 to completeswitching of the imaging optical path. In the timing chart TC10 shown inFIG. 7, because the third period T3 is the same as the first period T1,the switching unit 25 starts switching of the imaging optical pathduring the first period T1 and completes the switching of the imagingoptical path during the first period T1.

During the first period T1, the imaging device 22 reads pixel signals ofone frame in a time period which is less than or equal to half a lengthof a sum of the exposure period and the reading period of the cells 54of a row of each frame period of the plurality of frame periods. Thatis, the imaging device 22 reads pixel signals of one frame in a timeperiod which is less than or equal to half the length of the frameperiod. By shortening a time period required for reading the pixelsignals of the cells 54, the imaging device 22 can lengthen a period (asimultaneous exposure period) during which at least some of theplurality of cells 54 can be simultaneously exposed. Thus, thesensitivity of imaging is improved. In the timing chart TC10 shown inFIG. 7, the frame period is about 1/60 sec and the simultaneous exposureperiod corresponding to all the cells 54 is about 1/120 sec.

A time period in which the imaging device 22 reads the pixel signalsfrom the cells 54 of one row is less than or equal to a time period Tr.The time period Tr is represented by the following Eq. (1). A number mis an integer of 2 or more which is the number of simultaneous exposurerows of the imaging device 22. The simultaneous exposure period of cells54 of m rows includes all of the second period T2 during which the whitelight source 37 is continuously turned on. In Eq. (1), a time period Tfis a length of the frame period.

Tr=Tf/(m×2)  (1)

The second period T2 does not overlap the third period during whichswitching of the imaging optical path is executed. While the white lightsource 37 is turned on, the imaging optical path is fixed. That is,while the plurality of cells 54 are being exposed, the imaging opticalpath is fixed. An image output from the imaging device 22 is generatedon the basis of only one of the first optical image and the secondoptical image.

The measurement processing unit 347 executes a measurement process onthe basis of the first image and the second image. The first image isacquired during the first frame period and the second image is acquiredduring the second frame period. The first frame period and the secondframe period are two consecutive frame periods. An interval between afirst acquisition timing and a second acquisition timing is the same asone frame period. The first acquisition timing is a timing at which theimaging device 22 acquires the first image. The second acquisitiontiming is a timing at which the imaging device 22 acquires the secondimage.

FIG. 8 shows another example of relationships between an operation ofthe imaging device 22 in the measurement mode, a state of illumination,and a state of an imaging condition. The operation of the imaging device22 will be described with reference to FIG. 8. Descriptions of partsthat are the same as those shown in FIG. 7 will be omitted.

A timing chart TC11 shows the operation of the imaging device 22. Thelength of the second period T2 is different in the timing chart TC10 andthe timing chart TC11.

The white light source 37 is turned on after the completion of readingof pixel signals during one frame period. That is, the white lightsource 37 is turned on after completion of the first period T1. Thewhite light source 37 is turned off before the start of reading of pixelsignals during one frame period. That is, the white light source 37 isturned off before the start of the first period T1.

The second period T2 during which the white light source 37 iscontinuously turned on is a part of a period other than the first periodT1. In the timing chart TC11 shown in FIG. 8, the sum of the length ofthe first period T1 and the length of the second period T2 is shorterthan the length of the frame period.

The third period T3 during which the white light source 37 iscontinuously turned off is all of the period other than the secondperiod T2. In the timing chart TC11 shown in FIG. 8, the third period T3is different from the first period T1.

The second period T2 is started after the reading period of the rowfinally scanned by the imaging device 22 during one frame period iscompleted. The second period T2 is completed before the reading periodof the row first scanned by the imaging device 22 during one frameperiod is started. The difference between the timing of the start of thesecond period T2 and the timing of the end of the second period T2between the timing chart TC10 shown in FIG. 7 and the timing chart TC11shown in FIG. 8 (the length of the second period T2) is a result ofautomatic exposure (AE) control to be executed by the video processingunit 31.

FIG. 9 shows a procedure of the operation of the endoscope device 1. Theoperation of the endoscope device 1 will be described with reference toFIG. 9.

When the endoscope device 1 has been activated, the endoscope device 1operates in the observation mode. When the endoscope device 1 has beenactivated, initial setting is executed. In the initial setting, theimaging control unit 30 sets simultaneous exposure rows in the imagingdevice 22. The simultaneous exposure rows include cells 54 to besimultaneously exposed in accordance with the turning-on of the whitelight source 37. In the timing chart TC10 shown in FIG. 7 and the timingchart TC11 shown in FIG. 8, all the rows in the array of the pluralityof cells 54 are simultaneously exposed. In the initial setting, theimaging control unit 30 sets the length of the frame period in theimaging device 22. In the initial setting, the imaging control unit 30sets a length of the first period T1 during which the imaging device 22reads the pixel signals in the imaging device 22. The illuminationcontrol unit 33 causes the white light source 37 to be turned on (stepS100).

After step S100, the switching control unit 36 outputs a control signalfor switching the imaging optical path to the switching unit 25.Thereby, the switching control unit 36 causes the switching unit 25 tostart switching of the imaging optical path. The switching unit 25starts switching from the second optical path L2 to the first opticalpath L1 on the basis of the control signal from the switching controlunit 36. Thereafter, the switching control unit 36 causes the switchingunit 25 to complete the switching of the imaging optical path (stepS105). When the imaging optical path is already the first optical pathL1 in step S105, the processing in step S105 is unnecessary.

After step S105, the imaging device 22 generates an image of one frameand outputs the generated image. The imaging device 22 consecutivelyscans the cells 54 of the plurality of rows row by row during each frameperiod and sequentially reads the pixel signals from the cells 54 of theplurality of rows during each frame period. During each frame period,the imaging device 22 outputs an image based on the pixel signals of thecells 54 of the plurality of rows. Because the imaging optical path isthe first optical path L1, the imaging device 22 outputs the firstimage. The video processing unit 31 processes the first image outputfrom the imaging device 22. The first image processed by the videoprocessing unit 31 is output to the display processing unit 341 and theCPU 34 (step S110).

After step S110, the display processing unit 341 causes the display unit5 to display the first image generated in step S110. The display unit 5displays the first image (step S115).

After step S115, the control unit 340 determines whether or not toexecute measurement (step S120). For example, when the user has input ameasurement instruction by operating the operation unit 4, the controlunit 340 determines to perform measurement in step S120. The measurementmay be executed at a predetermined cycle. For example, when apredetermined time period has elapsed from the activation of theendoscope device 1, the control unit 340 determines to execute themeasurement. Alternatively, when a predetermined time period has elapsedfrom the execution of the previous measurement, the control unit 340determines to execute the measurement.

When the control unit 340 determines not to execute the measurement instep S120, the processing in step S105 is executed. Until a measurementinstruction is input, the imaging optical path is maintained as thefirst optical path L1. Until the measurement instruction is input, theimaging device 22 sequentially outputs first images and the display unit5 sequentially updates and displays the first images.

When the control unit 340 determines to execute the measurement in stepS120, the endoscope device 1 operates in the measurement mode. Theimaging device 22 generates an image of one frame and outputs thegenerated image. Because the imaging optical path is the first opticalpath L1, the imaging device 22 outputs the first image. The illuminationcontrol unit 33 causes the white light source 37 to be turned on duringthe second period. During the third period, the illumination controlunit 33 causes the white light source 37 to be turned off (step S130).

After step S130, the switching control unit 36 outputs a control signalfor switching the imaging optical path to the switching unit 25.Thereby, the switching control unit 36 causes the switching unit 25 tostart switching of the imaging optical path. The switching unit 25starts switching from the first optical path L1 to the second opticalpath L2 on the basis of the control signal from the switching controlunit 36. Thereafter, the switching control unit 36 causes the switchingunit 25 to complete switching of the imaging optical path (step S132).Actually, while the imaging device 22 is reading the pixel signals instep S130, the processing in step S132 is executed.

After step S132, the display processing unit 341 causes the display unit5 to display the first image generated in step S130. The display unit 5displays the first image (step S135).

After step S135, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging optical path is thesecond optical path L2, the imaging device 22 outputs a second image.The illumination control unit 33 causes the white light source 37 to beturned on during the second period. During the third period, theillumination control unit 33 causes the white light source 37 to beturned off (step S145).

After step S145, the switching control unit 36 outputs a control signalfor switching the imaging optical path to the switching unit 25.Thereby, the switching control unit 36 causes the switching unit 25 tostart switching of the imaging optical path. The switching unit 25starts switching from the second optical path L2 to the first opticalpath L1 on the basis of the control signal from the switching controlunit 36. Thereafter, the switching control unit 36 causes the switchingunit 25 to complete switching of the imaging optical path (step S147).Actually, while the imaging device 22 is reading the pixel signal instep S145, the processing in step S147 is executed.

After step S147, the control unit 340 determines whether or not theacquisition of a predetermined number of image sets has been completed.One image set is a combination of one first image and one second image.The first image included in the image set and the second image includedin the image set are acquired during consecutive frame periods. Forexample, the predetermined number is 2 (step S150).

When the control unit 340 determines that the acquisition of apredetermined number of image sets has not been completed in step S150,the processing in step S130 is executed. The imaging optical path isiteratively switched between the first optical path L1 and the secondoptical path L2 until a predetermined number of image sets are acquired.The imaging device 22 iteratively outputs first images and secondimages. When the first image and the second image mutually havingparallaxes are alternately displayed, visibility deteriorates. Thus, thedisplay unit 5 continuously sequentially updates and displays the firstimages without displaying the second images.

When the control unit 340 determines that the acquisition of apredetermined number of image sets has been completed in step S150, theimage processing unit 342 executes image processing (step S155).

After step S155, the control unit 340 determines whether or not theimage processing has succeeded on the basis of a processing resultgenerated in step S155 (step S160). When the control unit 340 determinesthat the image processing has succeeded in step S160, the processing instep S105 is executed. When the control unit 340 determines that theimage processing has failed in step S160, the processing in step S130 isexecuted.

The image output from the imaging device 22 may not be processed.Therefore, step S155 is optional. The display unit 5 may not display animage. Therefore, steps S115 and S135 are optional.

One image set may be acquired. Therefore, after step S145, theprocessing in step S155 may be executed. In this case, the processing instep S150 is not executed.

After step S105, the processing in step S130 may be executed. In thiscase, the processing in steps S110 to S120 is not executed.

FIG. 10 shows details of the image processing in step S155. Theoperation of the endoscope device 1 will be described with reference toFIG. 10.

An image set A and an image set B are input to the CPU 34. Each of imageset A and image set B includes one first image and one second image.

The suitability determination unit 345 calculates a determinationparameter for determining whether or not the image set is suitable forimage processing of a subsequent stage. Specifically, the suitabilitydetermination unit 345 executes a motion detection process. In themotion detection process, the suitability determination unit 345calculates a first value indicating first motion between two frames onthe basis of a first image of a set A and a first image of a set B. Inthe motion detection process, the suitability determination unit 345calculates a second value indicating second motion between two frames onthe basis of a second image of the set A and a second image of the setB. For example, the suitability determination unit 345 calculates anabsolute value of the difference between pixel values of two images foreach pixel of the image. The suitability determination unit 345calculates a sum of absolute values of differences between pixel valuesof all pixels of the image. The suitability determination unit 345calculates a first value and a second value as determination parameters.The first value is a sum of absolute values of differences between pixelvalues calculated from two first images. The second value is a sum ofabsolute values of differences between pixel values calculated from twosecond images. The motion detection method may be another method such asmotion vector calculation. Other than the motion detection process, aprocess of calculating the determination parameters in the suitabilitydetermination process may be another method such as a contrast detectionprocess or a brightness determination process. Alternatively, theprocess of calculating the determination parameters in the suitabilitydetermination process may be a combination thereof (step S200).

After step S200, the suitability determination unit 345 determineswhether or not the image set is suitable for image processing of thesubsequent stage on the basis of the calculated parameters. Thesuitability determination unit 345 determines whether or not eachdetermination parameter is less than a predetermined value. When thesuitability determination unit 345 determines that all determinationparameters are less than the predetermined value, the suitabilitydetermination unit 345 determines that the image set is suitable (hassuitability) for the image processing of the subsequent stage. When thesuitability determination unit 345 determines that at least one of thedetermination parameters is greater than or equal to the predeterminedvalue, the suitability determination unit 345 determines that the imageset is not suitable (does not have suitability) for the image processingof the subsequent stage (step S205).

If the suitability determination unit 345 determines that there is nosuitability in step S205, the control unit 340 stores a failure as aprocessing result (step S210). When the suitability determination unit345 determines that there is no suitability, the image processing unit342 stops the image processing. Specifically, the noise reduction unit346 stops the noise reduction process, the measurement processing unit347 stops the measurement process, and the visualization processing unit343 stops the visualization process. After step S210, the processing instep S160 is executed.

When the suitability determination unit 345 determines that there issuitability in step S205, the noise reduction unit 346 executes a noisereduction process. In the noise reduction process, the noise reductionunit 346 generates a first NR image (a third image) on the basis of thefirst image of the set A and the first image of the set B. A pixel valueof the first NR image is an addition average of the pixel values betweenthe two first images. In the noise reduction process, the noisereduction unit 346 generates a second NR image (a fourth image) on thebasis of the second image of the set A and the second image of the setB. A pixel value of the second NR image is an addition average of thepixel values between the two second images (step S215).

After step S215, the measurement processing unit 347 executes ameasurement process. In the measurement process, the measurementprocessing unit 347 generates a first corrected image by correctingoptical distortion of the first NR image. In the measurement process,the measurement processing unit 347 generates a second corrected imageby correcting the optical distortion of the second NR image. In themeasurement process, the measurement processing unit 347 acquires adisparity distribution in a template matching process using the first NRimage and the second NR image. The disparity distribution includesdisparity information in each pixel of a plurality of pixelsconstituting an image. In the measurement process, the measurementprocessing unit 347 calculates 3D coordinates of each point on thesurface of the subject by the principle of triangulation on the basis ofthe disparity distribution. 3D coordinates are calculated in each pixelof the plurality of pixels. The measurement processing unit 347generates color 3D data. The color 3D data includes color information ofeach pixel of the first NR image and 3D coordinates corresponding toeach pixel. The color information includes a red pixel value, a greenpixel value, and a blue pixel value. In the color 3D data, the colorinformation and the 3D coordinates are associated with each pixel. Thecolor 3D data may include color information of each pixel of the secondNR image and 3D coordinates corresponding to each pixel (step S220).

After step S220, the visualization processing unit 343 executes avisualization process. In the visualization process, the visualizationprocessing unit 343 generates a perspective projection image bydisposing the color 3D data within a virtual space. The perspectiveprojection image is equivalent to an image generated when a virtualcamera disposed within the virtual space has imaged a subject within thevirtual space. The CPU 34 outputs the generated perspective projectionimage to the video processing unit 31. The video processing unit 31outputs the perspective projection image to the display unit 5 (stepS225).

After step S225, the display processing unit 341 causes the display unit5 to display the perspective projection image generated in step S225.The display unit 5 displays the perspective projection image. Thedisplay processing unit 341 may cause the display unit 5 to display theperspective projection image instead of the first image or may cause thedisplay unit 5 to simultaneously display the first image and theperspective projection image (step S230).

After step S230, the control unit 340 stores a success as a processingresult (step S235). After step S235, the processing in step S160 isexecuted.

In step S205, the image processing unit 342 determines whether or notmotion has been detected on the basis of at least one of the first valueand the second value. The first value indicates motion in the pluralityof first images. The second value indicates motion in the plurality ofsecond images. When no motion has been detected, the image processingunit 342 executes the noise reduction process in step S215. When nomotion has been detected, the measurement processing unit 347 executesthe measurement process in step S220. When no motion has been detected,the visualization processing unit 343 executes the visualization processin step S225.

The suitability determination unit 345 may calculate only either one ofthe first value and the second value in the motion detection process.The suitability determination unit 345 may determine whether or notmotion has been detected on the basis of only either one of the firstvalue and the second value.

The imaging device 22 may generate a plurality of first images and onesecond image. In this case, the suitability determination unit 345calculates the first value on the basis of the plurality of firstimages. The imaging device 22 may generate one first image and aplurality of second images. In this case, the suitability determinationunit 345 calculates the second value on the basis of the plurality ofsecond images.

The image processing unit 342 may be configured to process three or moreimage sets and select the most appropriate set for image processing.Even if motion is detected between two images of two consecutive sets,there is a possibility that motion will not be detected between twoimages of other two consecutive sets. Thus, a probability of redoing ofimaging is lowered by selecting an object to be processed from the threeor more image sets. The noise reduction unit 346 can further reducenoise by executing the noise reduction process on the basis of three ormore images.

The suitability determination unit 345 is optional. Therefore, themotion detection processing in step S200 may not be executed. Thedetermination in step S205 may not be executed.

The noise reduction unit 346 is optional. Therefore, the noise reductionprocess in step S215 may not be executed. In this case, the measurementprocessing unit 347 executes a measurement process using one image set.The measurement processing unit 347 calculates 3D coordinates of aplurality of points on the surface of the subject using the passivestereo method on the basis of the first image corresponding to the firstoptical image and the second image corresponding to the second opticalimage.

One image set may be selected from a plurality of image sets and themeasurement processing unit 347 may execute the measurement processusing the selected image set. For example, the selected image setincludes two images having smaller values indicating motion. Theselected image set may include two images having higher contrast.

The visualization processing unit 343 is optional. Therefore, thevisualization process in step S225 may not be executed.

The display unit 5 may display a value indicating whether or not theimage is suitable for image processing, i.e., a result of calculating avalue indicating motion. When the image is not suitable for imageprocessing, i.e., only when motion has been detected, the display unit 5may display the result of calculating the value indicating motion. Forexample, the calculation result is a message corresponding to a valueindicating motion. For example, the message indicates that motion hasbeen detected. When motion has been detected, the endoscope device 1 cannotify the user of a warning.

The display of the image in the measurement mode is optional. Therefore,the first image may not be displayed in steps S110 and S135. Theperspective projection image may not be displayed in step S230.

The processes to be executed by the image processing unit 342, thevisualization processing unit 343, and the display processing unit 341may be executed outside the endoscope device 1. That is, the aboveprocess may be executed by an external computer system such as a PC or anetwork server connected to the endoscope device 1 in a wireless orwired manner.

The first embodiment of the present invention is compared with areference form of the present invention with reference to FIG. 26. Theendoscope device 1 does not need to execute an operation in thereference form. Hereinafter, the operation of the endoscope device 1 inthe reference form will be described for comparison between the firstembodiment and the reference form.

A timing chart TC100 shown in FIG. 26 shows the operation of the imagingdevice 22. Descriptions of parts that are the same as those shown inFIG. 7 will be omitted. The white light source 37 is continuously turnedon.

Before the reading period of the cells 54 of the first row during theframe period (i−1) is started, the imaging optical path is the secondoptical path L2. When the exposure period of the cells 54 of the firstrow during the frame period (i−1) has ended, the reading period of thecells 54 of the first row during the frame period (i−1) is started. Atthis moment, switching of the imaging optical path is started. Theswitching unit 25 starts switching from the second optical path L2 tothe first optical path L1.

When the reading period of the cells 54 of the first row during theframe period (i−1) has ended, the switching unit 25 completes switchingfrom the second optical path L2 to the first optical path L1. Theimaging optical path is the first optical path L1.

When the exposure period of the cells 54 of the first row during theframe period (i+1) has ended, the reading period of the cells 54 of thefirst row during the frame period (i+1) is started. At this moment,switching of the imaging optical path is started. The switching unit 25starts switching from the first optical path L1 to the second opticalpath L2.

When the reading period of the cells 54 of the first row during theframe period (i+1) has ended, the switching unit 25 completes switchingfrom the first optical path L1 to the second optical path L2. Theimaging optical path is the second optical path L2.

When the exposure period of the cells 54 of the first row during theframe period (i+3) has ended, the reading period of the cells 54 of thefirst row during the frame period (i+3) is started. At this moment,switching of the imaging optical path is started. The switching unit 25starts switching from the second optical path L2 to the first opticalpath L1.

When the reading period of the cells 54 of the first row during theframe period (i+3) has ended, the switching unit 25 completes theswitching from the second optical path L2 to the first optical path L1.The imaging optical path is the first optical path L1.

In FIG. 26, an image (an output image) output from the imaging device 22is schematically shown. An image IMG100 includes pixel signals read fromcells 54 of eight rows during a reading period in the frame period(i−1). Switching from the second optical path L2 to the first opticalpath L1 is executed during an exposure period in the frame period (i−1).The image IMG100 includes pixel signals based on the first optical imageand pixel signals based on the second optical image. Therefore, theimage IMG100 is not suitable for measurement.

The image IMG101 includes pixel signals read from the cells 54 of theeight rows during a reading period in a frame period i. Switching of theimaging optical path is not executed during the exposure period in theframe period i. The image IMG101 includes pixel signals based on thefirst optical image.

An image IMG102 includes pixel signals read from the cells 54 of theeight rows during the reading period in the frame period (i+1).Switching from the first optical path L1 to the second optical path L2is executed during the exposure period in the frame period (i+1). Theimage IMG102 includes pixel signals based on the first optical image andpixel signals based on the second optical image. Therefore, the imageIMG102 is not suitable for measurement.

An image IMG103 includes pixel signals read from the cells 54 of theeight rows during the reading period in the frame period (i+2).Switching of the imaging optical path is not executed during theexposure period in the frame period (i+2). The image IMG103 includespixel signals based on the second optical image.

The measurement processing unit 347 executes a measurement process onthe basis of the image IMG101 and the image IMG103. An interval betweena first acquisition timing and a second acquisition timing is the sameas a period of two frames. The first acquisition timing is a timing atwhich the imaging device 22 acquires the image IMG101. The secondacquisition timing is a timing at which the imaging device 22 acquiresthe image IMG103.

In the timing chart TC100 shown in FIG. 26, an interval at which twoimages for measurement are acquired is the same as a period of twoframes. On the other hand, in the timing chart TC10 shown in FIG. 7, aninterval at which two images for measurement are acquired is the same asa period of one frame. In the timing chart TC10 shown in FIG. 7,compared with the timing chart TC100 shown in FIG. 26, an interval atwhich two images for measurement are acquired is short.

A method of operating an endoscope device according to each aspect ofthe present invention includes a first step, a second step, and a thirdstep. In the first step (S130 and S145), the illumination control unit33 causes the white light source 37 to generate illumination lightduring the second period. In the second step (S130 and S145), theillumination control unit 33 causes the white light source 37 to stopthe generation of the illumination light during the third period. In thethird step (S132 and S147), the switching control unit 36 causes theswitching unit 25 to start switching of the imaging condition during thethird period and complete the switching of the imaging condition duringthe third period. The switching control unit 36 causes the switchingunit 25 to complete switching of the imaging condition before the nextsecond period is started. That is, the switching control unit 36 causesthe switching unit 25 to complete the switching of the imaging conditionduring the third period which is the same as the third period duringwhich switching of the imaging conditions has been started.

The endoscope device 1 can shorten a time interval of imaging under aplurality of imaging conditions. Thus, a time interval at which aplurality of images for image processing are acquired is shortened.Motions between a plurality of images used for image processing arereduced and deterioration in the quality of an image processing resultdue to an influence of motion is reduced. Furthermore, because theendoscope device 1 performs global exposure, deterioration in thequality of the image processing result due to an influence of rollingdistortion is reduced. That is, the quality of the image processingresult is improved. Thus, measurement accuracy is improved in themeasurement process.

When an image with motion is used for the measurement process,measurement accuracy deteriorates. Only when the motion has not beendetected, the measurement processing unit 347 executes the measurementprocess. Therefore, measurement accuracy is improved.

When an image with motion has been used for the visualization process,the quality of the result obtained by the reconstruction of a 3D shapeof a subject deteriorates. Only when no motion has been detected, thevisualization processing unit 343 executes the visualization process.Thus, the quality of the result obtained by the reconstruction of the 3Dshape of the subject is improved.

The noise reduction unit 346 executes the noise reduction process usinga plurality of image sets. Thereby, the accuracy of template matchingprocess is improved. Thus, the measurement accuracy is improved and thequality of the result obtained by the reconstruction of the 3D shape ofthe subject is improved.

Second Embodiment

A second embodiment of the present invention will be described using theendoscope device 1 shown in FIG. 2. The imaging device 22 captures animage at high speed (in a shorter frame period). When imaging isperformed at high speed, some of the cells 54 are not properly exposed.In the image, the quality of the region corresponding to the cell 54which is not properly exposed deteriorates. In the second embodiment,the quality of the image that is displayed is maintained by limiting arange of the image that is displayed on the display unit 5.

FIG. 11 shows relationships between an operation of the imaging device22 in a measurement mode, a state of illumination, and a state of animaging condition. The operation of the imaging device 22 will bedescribed with reference to FIG. 11. Descriptions of parts that are thesame as those shown in FIG. 7 will be omitted.

A timing chart TC12 shows an operation of the imaging device 22. In thetiming chart TC12, a frame rate is 120 fps. A length of each frameperiod is 1/120 sec. A time period during which the imaging device 22reads pixel signals from cells 54 of one row is a time period Tr. Thetime period Tr is represented by the following Eq. (2). A number m is aninteger of 2 or more. An exposure period of cells 54 of m rows overlapsa second period during which the white light source 37 is continuouslyturned on.

Tr=1/(120×m)  (2)

Reading periods of cells 54 of first to eighth rows during a frameperiod (i−1) are sequentially started and pixel signals of the cells 54of the first to eighth rows are sequentially read. The cells 54 of thefirst to eighth rows are sequentially reset and exposure periods of thecells 54 of the first to eighth rows during a frame period i aresequentially started.

When the reading period of the cells 54 of the first row during theframe period (i−1) has ended, the white light source 37 is alreadyturned on. When the reading period of the cells 54 of the first rowduring the frame period (i−1) has ended, an imaging optical path is asecond optical path L2.

When the reading period of the cells 54 of the second row during theframe period (i−1) has ended, the white light source 37 is turned off.

When the reading period of the cells 54 of the fourth row during theframe period (i−1) has been started, switching of the imaging opticalpath is started. The switching unit 25 starts switching from the secondoptical path L2 to a first optical path L1.

When the reading period of the cells 54 of the fourth row during theframe period (i−1) has ended, the switching unit 25 completes switchingfrom the second optical path L2 to the first optical path L1. Theimaging optical path is the first optical path L1.

When the reading period of the cells 54 of the sixth row during theframe period (i−1) has ended, the white light source 37 is turned on.

When the reading period of the cells 54 of the eighth row during theframe period (i−1) has ended, the reading period of the cells 54 of thefirst row during the frame period i is started. The reading periods ofthe cells 54 of the first to eighth rows during the frame period i aresequentially started and the pixel signals of the cells 54 of the firstto eighth rows are sequentially read. The cells 54 of the first toeighth rows are sequentially reset and the exposure periods of the cells54 of the first to eighth rows during a frame period (i+1) aresequentially started.

When the reading period of the cells 54 of the second row during theframe period i has ended, the white light source 37 is turned off.

When the reading period of the cells 54 of the fourth row during theframe period i has been started, switching of the imaging optical pathis started. The switching unit 25 starts switching from the firstoptical path L1 to the second optical path L2.

When the reading period of the cells 54 of the fourth row during theframe period i has ended, the switching unit 25 completes the switchingfrom the first optical path L1 to the second optical path L2. Theimaging optical path is the second optical path L2.

Thereafter, the white light source 37 is periodically turned on. Theswitching unit 25 periodically switches the imaging optical path.

In the timing chart TC12, during the second period T2, cells 54 ofsimultaneous exposure rows which are some rows consecutively disposedamong a plurality of rows in an array of the plurality of cells 54 aresimultaneously exposed. A period during which the cells 54 are exposedby turning on the white light source 37 is common to the cells 54 of thesimultaneous exposure rows. The cells 54 of the third to sixth rowswhich are the simultaneous exposure rows are simultaneously exposed. Thesecond period T2 is common to the cells 54 of the third to sixth rows.The second period T2 does not include the reading periods of the cells54 of the third to sixth rows.

The second period T2 includes reading periods of the cells 54 of thefirst row and the second row of a certain frame period and the seventhrow and the eighth row of a frame period previous to the certain frameperiod. A period during which the cells 54 of these rows are exposed byturning on the white light source 37 is shorter than exposure timeperiods of the cells 54 of the third to sixth rows. Alternatively, boththe first optical image and the second optical image are exposed.Accordingly, the cells 54 of the first row, the second row, the seventhrow, and the eighth row are not properly exposed.

The simultaneous exposure rows are consecutively disposed in half theplurality of rows in the array of the plurality of cells 54. Thesimultaneous exposure rows include cells 54 to be simultaneously exposedby turning on the white light source 37. In an upper quarter of theplurality of rows and a lower quarter of the plurality of rows, thecells 54 are not properly exposed.

An image IMG10 includes pixel signals read during the reading period inthe frame period (i−1). An image IMG11 includes pixel signals readduring the reading period in the frame period i. An image IMG12 includespixel signals read during the reading period in the frame period (i+1).An image IMG13 includes pixel signals read during the reading period inthe frame period (i+2).

The image IMG10 and the image IMG12 are generated on the basis of thesecond optical image. The image IMG11 and the image IMG13 are generatedon the basis of the first optical image. Central regions of the imageIMG10, the image IMG11, the image IMG12, and the image IMG13 includepixel signals read from the simultaneously exposed cells 54. Regions ofupper quarters of the image IMG10, the image IMG11, the image IMG12, andthe image IMG13 include pixel signals read from the cells 54 which arenot sufficiently exposed. Regions of lower quarters of the image IMG10,the image IMG11, the image IMG12, and the image IMG13 include pixelsignals read from the cells 54 which are not sufficiently exposed.Alternatively, the regions of the upper and lower quarters of the imageIMG10, the image IMG11, the image IMG12, and the image IMG13 includepixel signals read from the pixels 54 exposed on the basis of both thefirst optical image and the second optical image.

The video processing unit 31 executes electronic mask processing. In theelectronic mask processing, the video processing unit 31 executes aprocess of preventing an image of a region corresponding to rows otherthan the simultaneous exposure rows from being displayed. For example,the video processing unit 31 superimposes a black graphic in the regioncorresponding to the rows other than the simultaneous exposure rows inthe image. In the image displayed by the display unit 5, the regioncorresponding to the rows other than the simultaneous exposure rows isblack.

The video processing unit 31 may replace pixel values of the regioncorresponding to the rows other than the simultaneous exposure rows inthe image with the black value. Also in this case, in the imagedisplayed by the display unit 5, the region corresponding to the rowsother than the simultaneous exposure rows is black.

The video processing unit 31 may clip only the image of the regioncorresponding to the row other than the simultaneous exposure rows fromthe images corresponding to the plurality of rows. In this case, theimage displayed by the display unit 5 includes only the image of theregion corresponding to the simultaneous exposure rows.

The simultaneous exposure rows may be changed in accordance with anindication from the user. For example, the user inputs an indicationindicating a position of the simultaneous exposure rows by operating theoperation unit 4. The imaging control unit 30 sets the simultaneousexposure rows in the imaging device 22 on the basis of the positionsindicated by the user.

When the simultaneous exposure rows are changed, the first period T1 ischanged. By changing the first period T1, the second period T2 and thethird period T3 are changed. The imaging optical paths are switchedwithin the third period T3. When the third period T3 is changed, atiming at which the imaging optical paths are switched may be changed insome cases.

In the image, a region corresponding to the cells 54 which are notsufficiently exposed is not displayed. Therefore, the quality of theimage displayed by the display unit 5 is maintained.

In the image, the range in which the pixel signals based on the opticalimage of the subject are included is narrowed. Because the frame periodbecomes short, the endoscope device 1 can shorten a time interval ofimaging under a plurality of imaging conditions more than in the firstembodiment. As a result, the quality of an image processing result in ascene in which there is motion of the camera is improved as comparedwith the first embodiment.

Third Embodiment

FIG. 12 shows a configuration of an endoscope device 1 a according to athird embodiment of the present invention. Descriptions of parts thatare the same as those shown in FIG. 2 will be omitted.

The main body unit 3 shown in FIG. 2 is changed to a main body unit 3 a.The main body unit 3 a does not have the switching control unit 36. Inthe main body unit 3 a, the light source unit 32 shown in FIG. 2 ischanged to a light source unit 32 a. The light source unit 32 a has awhite light source 37 and a laser diode (LD) 38. In the main body unit 3a, the illumination control unit 33 shown in FIG. 2 is changed to anillumination control unit 33 a.

The optical adapter 21 shown in FIG. 2 is changed to an optical adapter21 a. In the optical adapter 21 a, the observation optical system 60shown in FIG. 2 is changed to an observation optical system 60 a. Theobservation optical system 60 a includes a concave lens 23 and a convexlens 24.

The illumination optical system 80 shown in FIG. 2 is changed to anillumination optical system 80 a. The optical adapter 21 a has a part ofthe illumination optical system 80 a. The illumination optical system 80a includes a condenser lens 81, a light guide 82, a rod lens 83, adiffusion lens 84, a fiber 85, a collimator lens 86, a rod lens 87, anda diffractive optical element 88. The condenser lens 81 is disposed inthe main body unit 3 a. The light guide 82 and the fiber 85 are disposedin the main body unit 3 a and the insertion unit 20. The rod lens 83,the diffusion lens 84, the collimator lens 86, the rod lens 87, and thediffractive optical element 88 are disposed in the optical adapter 21 a.

The outline of parts different from those of FIG. 2 will be described.The endoscope device 1 a has the white light source 37 (a first lightsource) and the LD 38 (a second light source) as light sources. Theillumination light includes first illumination light and secondillumination light. The white light source 37 generates white light asthe first illumination light. The LD 38 generates the secondillumination light. The illumination optical system 80 a has adiffractive optical element 88 (a pattern generation unit). Thediffractive optical element 88 (the pattern generation unit) gives aspatial pattern including a bright part and a dark part to the secondillumination light. The illumination optical system 80 a radiates thesecond illumination light to which the pattern is given to the subject.

The plurality of imaging conditions include a first imaging conditionand a second imaging condition. Under the first imaging condition, thefirst illumination light is radiated to the subject and the irradiationof the second illumination light to the subject is stopped. Under thesecond imaging condition, the second illumination light is radiated tothe subject and the radiation of the first illumination light to thesubject is stopped. The imaging device 22 generates a first image of thesubject by imaging the subject under the first imaging condition. Theimaging device 22 generates a second image of the subject by imaging thesubject under the second imaging condition.

The measurement processing unit 347 calculates 3D coordinates of aplurality of points on the surface of the subject using an active stereomethod on the basis of the second image. The imaging device 22 has a redcolor filter, a green color filter, and a blue color filter. The imagingdevice 22 generates a color image as the first image. The color imageincludes a plurality of pixels. Each pixel has information indicatingeach of brightness of red, brightness of green, and brightness of blueas a pixel value. The image processing unit 342 generates 3D shape datain which the 3D coordinates of the plurality of points are associatedwith pixel values of the plurality of points. The visualizationprocessing unit 343 generates graphic data in which the 3D shape data isvisualized.

The imaging device 22 generates a plurality of at least one of firstimages of the subject and second images of the subject. The suitabilitydetermination unit 345 of the CPU 34 calculates at least one of a valueindicating whether or not a plurality of first images are suitable forimage processing and a value indicating whether or not a plurality ofsecond images are suitable for image processing.

The imaging device 22 generates a plurality of second images by imagingthe subject under the second imaging condition. The noise reduction unit346 of the CPU 34 generates a third image by executing a noise reductionprocess on the plurality of second images. The noise reduction unit 346calculates 3D coordinates of a plurality of points on the surface of thesubject on the basis of the third image.

Details of parts different from those of FIG. 2 will be described. TheLD 38 generates blue laser light as second illumination light. The LD 38may be disposed in the optical adapter 21 a. The fiber 85 is a singlemode fiber. The fiber 85 is connected to the LD 38. The laser lightemitted from the LD 38 is transmitted to the tip of the insertion unit20 via the fiber 85. The laser light emitted from the fiber 85 isincident on the rod lens 87 as parallel light via the collimator lens86. The laser light is transmitted by the rod lens 87. The laser lightemitted from the rod lens 87 is incident on the diffractive opticalelement 88. The laser light emitted from the rod lens 87 is parallellight having a uniform phase. The diffractive optical element 88converts the parallel light into pattern light (the second illuminationlight) having a random spatial pattern due to an effect of interference.In the pattern light, the bright part and the dark part are spatiallyrandomly disposed. The pattern light emitted from the diffractiveoptical element 88 is radiated to the subject.

The observation optical system 60 a is a monocular optical system. Theobservation optical system 60 a takes in light reflected on the surfaceof the subject illuminated with white light or pattern light. Theconcave lens 23 and the convex lens 24 form an optical image based onthe light from the subject in an imaging region S1 of the imaging device22.

The illumination control unit 33 a controls the white light source 37and the LD 38. The illumination control unit 33 a causes the white lightsource 37 and the LD 38 to generate illumination light during a secondperiod. The illumination control unit 33 a causes the white light source37 and the LD 38 to stop the generation of the illumination light duringa third period. The illumination control unit 33 a functions as aswitching unit and a switching control unit. The illumination controlunit 33 a starts switching of the imaging condition during the thirdperiod and completes the switching of the imaging condition during thethird period.

The illumination control unit 33 a sets a first imaging condition duringa plurality of first frame periods. The illumination control unit 33 asets a second imaging condition during a plurality of second frameperiods. Each second frame period of the plurality of second frameperiods is different from each first frame period of the plurality offirst frame periods.

Under the first imaging condition, the illumination control unit 33 acauses the white light source 37 to be turned on and causes the LD 38 tobe turned off. Therefore, the illumination optical system 80 a radiatesthe white light to the subject under the first imaging condition. Theillumination optical system 80 a does not radiate the pattern light tothe subject under the first imaging condition. Under the first imagingcondition, the observation optical system 60 a takes in light reflectedon the surface of the subject illuminated with white light.

Under the second imaging condition, the illumination control unit 33 acauses the LD 38 to be turned on and causes the white light source 37 tobe turned off. Therefore, the illumination optical system 80 a radiatesthe pattern light to the subject under the second imaging condition. Theillumination optical system 80 a does not radiate white light to thesubject under the second imaging condition. Under the second imagingcondition, the observation optical system 60 a takes in the lightreflected on the surface of the subject illuminated with the patternlight.

FIG. 13 shows relationships between an operation of the imaging device22 in a measurement mode, a state of illumination, and a state of animaging condition. The operation of the imaging device 22 will bedescribed with reference to FIG. 13. Descriptions of parts that are thesame as those shown in FIG. 7 will be omitted.

A timing chart TC13 shows the operation of the imaging device 22. Cells54 of first to eighth rows are sequentially reset and exposure periodsof the cells 54 of the first to eighth rows during the frame period iare sequentially started.

When the exposure period of the cells 54 of the first row during theframe period i has been started, the white light source 37 and the LD 38are already turned off. When the exposure period of the cells 54 of thefirst row during the frame period i has been started, no imagingcondition is set.

When the exposure period of the cells 54 of the eighth row during theframe period i has been started, the turning-on of the white lightsource 37 is started. The illumination control unit 33 a causes thewhite light source 37 to be turned on by outputting a turning-on controlsignal to the white light source 37. Thereby, the illumination controlunit 33 a sets the imaging condition to the first imaging condition. Thewhite light source 37 starts turning-on on the basis of the controlsignal from the illumination control unit 33 a.

When the reading period of the cells 54 of the first row during theframe period i has been started, the turning-on of the white lightsource 37 ends. The illumination control unit 33 a causes the whitelight source 37 to be turned off by outputting a turning-off controlsignal to the white light source 37. The white light source 37 is turnedoff on the basis of the control signal from the illumination controlunit 33 a. Thereby, the illumination control unit 33 a starts switchingfrom the first imaging condition to the second imaging condition.

When the reading period of the cells 54 of the eighth row during theframe period i has ended, the turning-on of the LD 38 is started. Theillumination control unit 33 a causes the LD 38 to be turned on byoutputting a turning-on control signal to the LD 38. Thereby, theillumination control unit 33 a completes the switching from the firstimaging condition to the second imaging condition. The LD 38 startsturning-on on the basis of the control signal from the illuminationcontrol unit 33 a.

When the reading period of the cells 54 of the first row during theframe period (i+1) has been started, the turning-on of the LD 38 ends.The illumination control unit 33 a causes the LD 38 to be turned off byoutputting a turning-off control signal to the LD 38. The LD 38 isturned off on the basis of the control signal from the illuminationcontrol unit 33 a. Thereby, the illumination control unit 33 a startsswitching from the second imaging condition to the first imagingcondition.

When the reading period of the cells 54 of the eighth row during theframe period (i+1) has ended, the turning-on of the white light source37 is started. The illumination control unit 33 a causes the white lightsource 37 to be turned on by outputting a turning-on control signal tothe white light source 37. Thereby, the illumination control unit 33 acompletes the switching from the second imaging condition to the firstimaging condition. The white light source 37 starts turning-on on thebasis of the control signal from the illumination control unit 33 a.

The white light source 37 is turned on during the simultaneous exposureperiod in each of a frame period i and a frame period (i+2). The LD 38is turned on during the simultaneous exposure period in each of a frameperiod (i+1) and a frame period (i+3). The white light source 37 and theLD 38 are iteratively turned on and off. The white light source 37 andthe LD 38 are alternately turned on.

The illumination control unit 33 a iterates switching from the firstimaging condition to the second imaging condition and switching from thesecond imaging condition to the first imaging condition. Theillumination control unit 33 a executes switching of the imagingcondition for each frame period. Alternatively, the illumination controlunit 33 a may switch the imaging condition to the second imagingcondition every n frame periods.

During a first period T1, the imaging device 22 sequentially reads pixelsignals from at least some of the plurality of cells 54 row by row. In atiming chart TC13 shown in FIG. 13, the first period T1 is a period fromthe start of the reading of the pixel signals in the cells 54 of thefirst row to the completion of the reading of the pixel signals in thecells 54 of the eighth row.

The white light source 37 or the LD 38 is turned on simultaneously withthe completion of reading of the pixel signals during one frame period.That is, the white light source 37 or the LD 38 is turned onsimultaneously with the completion of the first period T1. The whitelight source 37 or the LD 38 is turned off simultaneously with the startof the reading of the pixel signals during one frame period. That is,the white light source 37 or the LD 38 is turned off simultaneously withthe start of the first period T1.

The illumination control unit 33 a causes the white light source 37 orthe LD 38 to generate illumination light during the second period T2.The white light source 37 or LD 38 is continuously turned on during thesecond period T2. In the timing chart TC13 shown in FIG. 13, the secondperiod T2 is at least a part of a period other than the first period T1.In the timing chart TC13 shown in FIG. 13, the second period T2 is allof the period other than the first period T1. A turning-on period of thewhite light source 37 may be different from a turning-on period of theLD 38.

The illumination control unit 33 a causes the white light source 37 orthe LD 38 to stop the generation of the illumination light during thethird period T3. The white light source 37 or the LD 38 is continuouslyturned off during the third period T3. The third period T3 is all of aperiod other than the second period T2. In the timing chart TC13 shownin FIG. 13, the third period T3 is the same as the first period T1. Thesum of the length of the third period T3 and the length of the secondperiod T2 is the same as the length of the frame period.

The illumination control unit 33 a starts switching of the imagingcondition during the third period T3 and completes the switching of theimaging condition during the third period T3. When the third period T3has been started, the illumination control unit 33 a starts theswitching of the imaging condition by causing the white light source 37or the LD 38 to be turned off. When the third period T3 ends, theillumination control unit 33 a completes the switching of the imagingcondition by causing the white light source 37 or the LD 38 to be turnedon. In the timing chart TC13 shown in FIG. 13, because the third periodT3 is the same as the first period T1, the illumination control unit 33a starts the switching of the imaging condition during the first periodT1 and completes the switching of the imaging condition during the firstperiod T1.

FIG. 14 shows a procedure of the operation of the endoscope device 1 a.The operation of the endoscope device 1 a will be described withreference to FIG. 14. Descriptions of processing that is the same asthat shown in FIG. 9 will be omitted.

After step S100, the illumination control unit 33 a sets the imagingcondition to observation illumination (a first imaging condition) bycausing the white light source 37 to be turned on (step S105 a).

After step S105 a, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging condition is theobservation illumination, the imaging device 22 outputs a first image.When the third period is started, the illumination control unit 33 acauses the white light source 37 to be turned off (step S110 a).

When the control unit 340 has determined to execute the measurement instep S120, the imaging device 22 generates an image of one frame andoutputs the generated image. Because the imaging condition is theobservation illumination, the imaging device 22 outputs a first image.When the third period is started, the illumination control unit 33 acauses the white light source 37 to be turned off (step S130 a).

After step S130 a, the illumination control unit 33 a sets the imagingcondition to pattern illumination (a second imaging condition) bycausing the LD 38 to be turned on (step S132 a). After step S132 a, theprocessing in step S135 is executed.

After step S135, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging condition ispattern illumination, the imaging device 22 outputs a second image. Whenthe third period starts, the illumination control unit 33 a causes theLD 38 to be turned off (step S145 a).

After step S145 a, the illumination control unit 33 a sets the imagingcondition to the observation illumination (the first imaging condition)by causing the white light source 37 to be turned on (step S147 a).After step S147 a, the processing in step S150 is executed.

When the control unit 340 determines that the acquisition of apredetermined number of image sets has not been completed in step S150,the processing in step S130 a is executed. Until a predetermined numberof image sets are acquired, the imaging condition is alternately anditeratively switched between the observation illumination and thepattern illumination. The imaging device 22 iteratively outputs thefirst image and the second image. Visibility deteriorates when thesecond image based on the pattern light is displayed. Thus, the displayunit 5 sequentially updates and displays the first image withoutdisplaying the second image. The pattern of switching the imagingconditions is not limited to the above-described pattern. The imagingdevice 22 may acquire second images after the imaging device 22 acquiresfirst images equal in number to a predetermined number of frames.

When the control unit 340 determines that the image processing hasfailed in step S160, the processing in step S130 a is executed.

The processing shown in FIG. 10 is changed to the following process. Animage set A and an image set B are input to the CPU 34. Each of theimage set A and image set B includes one first image and one secondimage generated from two consecutive frame periods.

In the noise reduction process (step S215), the noise reduction unit 346generates a second NR image (a third image) on the basis of the secondimage of the set A and the second image of the set B. A pixel value ofthe second NR image is an addition average of pixel values between thetwo second images.

In the measurement process (step S220), the measurement processing unit347 generates a second corrected image by correcting optical distortionof the second NR image. The measurement processing unit 347 stores areference pattern image. The reference pattern image is an image of aplane irradiated with the pattern light generated by the diffractiveoptical element 88. In the measurement process, the measurementprocessing unit 347 acquires a distribution of phase differences in atemplate matching process using the reference pattern image and thesecond NR image. The phase difference is a deviation amount of thepattern in the reference pattern image and the second NR image. Thedistribution of the phase differences includes phase differenceinformation in each cell 54 of the plurality of cells 54. In themeasurement process, the measurement processing unit 347 calculates 3Dcoordinates of each point on the surface of the subject by the principleof triangulation on the basis of the distribution of the phasedifferences. The measurement processing unit 347 generates color 3Ddata. The color 3D data includes color information of each pixel of thefirst image and 3D coordinates corresponding to each pixel.

The visualization processing unit 343 generates a perspective projectionimage in the visualization process (step S225). The CPU 34 outputs thegenerated perspective projection image to the video processing unit 31.The video processing unit 31 outputs the perspective projection image tothe display unit 5.

The endoscope device 1 a can shorten a time interval between the imagingof the subject illuminated with the white light and the imaging of thesubject irradiated with the pattern light. Thus, the quality of theimage processing result is improved. Measurement accuracy is improved inthe measurement process. A deviation of color texture with respect to areconstructed 3D shape of the subject decreases. Furthermore, becausethe endoscope device 1 a performs global exposure, it is possible toreduce deterioration in the quality of the image processing result dueto the influence of rolling distortion.

Second illumination light generated by the LD 38 may be laser lighthaving a wavelength other than that of blue. For example, the wavelengthof the second illumination light may be red, green, or infraredwavelength. Also, under the second imaging condition, radiation of firstillumination light may not be stopped.

Fourth Embodiment

FIG. 15 shows a configuration of an endoscope device 1 b according to afourth embodiment of the present invention. Descriptions of parts thatare the same as those shown in FIG. 12 will be omitted.

The main body unit 3 a shown in FIG. 12 is changed to a main body unit 3b. The main body unit 3 b includes a switching control unit 36 b inaddition to the configuration shown in FIG. 12.

The optical adapter 21 a shown in FIG. 12 is changed to an opticaladapter 21 b. The illumination optical system 80 a shown in FIG. 12 ischanged to an illumination optical system 80 b. The optical adapter 21 bhas a part of the illumination optical system 80 b. The illuminationoptical system 80 b includes a condenser lens 81, a light guide 82, arod lens 83, a diffusion lens 84, a fiber 85, and a stripe generationunit 89 (a pattern generation unit). The condenser lens 81 is disposedin the main body unit 3 b. The light guide 82 and the fiber 85 aredisposed in the main body unit 3 b and an insertion unit 20. The rodlens 83, the diffusion lens 84, and the stripe generation unit 89 aredisposed in the optical adapter 21 b.

FIG. 16 shows a configuration of the stripe generation unit 89 (thepattern generation unit). The stripe generation unit 89 shown in FIG. 16has a first optical path 890, a second optical path 891, a third opticalpath 892, and a phase shift unit 893 (a phase shifter).

The outline of parts different from those of FIG. 12 will be described.The stripe generation unit 89 gives a spatial pattern including a brightpart and a dark part to illumination light. A plurality of imagingconditions further include a third imaging condition. The imaging device22 generates a third image of a subject by imaging the subject under thethird imaging condition. The phase shift unit 893 functions as aswitching unit. The phase shift unit 893 performs switching between afirst imaging condition, a second imaging condition, and the thirdimaging condition by shifting the phase of the pattern of theillumination light. A phase of a pattern under the first imagingcondition, a phase of a pattern under the second imaging condition, anda phase of a pattern under the third imaging condition are differentfrom one another. The measurement processing unit 347 of the CPU 34calculates 3D coordinates of a plurality of points on the surface of thesubject using the phase shift method on the basis of at least the firstimage, the second image, and the third image.

The endoscope device 1 b includes a white light source 37 (a first lightsource) and an LD 38 (a second light source) as light sources. Theillumination light includes first illumination light and secondillumination light. The white light source 37 generates white light asthe first illumination light. The LD 38 generates the secondillumination light. The stripe generation unit 89 applies the pattern tothe second illumination light. The plurality of imaging conditionsfurther include a fourth imaging condition. Under the first imagingcondition, the second imaging condition, and the third imagingcondition, the second illumination light is radiated to the subject andradiation of the first illumination light to the subject is stopped.Under the fourth imaging condition, the first illumination light isradiated to the subject and radiation of the second illumination lightto the subject is stopped. The imaging device generates a fourth imageof the subject by imaging the subject under the fourth imagingcondition. The imaging device 22 has a red color filter, a green colorfilter, and a blue color filter. The imaging device 22 generates a colorimage as the fourth image. The color image includes a plurality ofpixels. Each pixel has information of each of brightness of red,brightness of green, and brightness of blue as pixel values. Themeasurement processing unit 347 generates 3D shape data in which 3Dcoordinates of a plurality of points are associated with pixel values ofa plurality of points. The visualization processing unit 343 generatesgraphic data in which the 3D shape data is visualized.

The imaging device 22 generates a plurality of at least one of firstimages, a second images, third images, and fourth images. Thesuitability determination unit 345 determines at least one of a valueindicating whether or not the plurality of first images are suitable forimage processing, a value indicating whether or not the plurality ofsecond images are suitable for image processing, a value indicatingwhether or not the plurality of third images are suitable for imageprocessing, and a value indicating whether or not the plurality offourth images are suitable for image processing.

The switching control unit 36 b causes the phase shift unit 893 to setthe first imaging condition during the plurality of first frame periods.The switching control unit 36 b causes the phase shift unit 893 to setthe second imaging condition during the plurality of second frameperiods. Each second frame period of the plurality of second frameperiods is different from each first frame period of the plurality offirst frame periods. The switching control unit 36 b causes the phaseshift unit 893 to set the third imaging condition during the pluralityof third frame periods. Each third frame period of the plurality ofthird frame periods is different from each first frame period of theplurality of first frame periods and is different from each second frameperiod of the plurality of second frame periods.

The noise reduction unit 346 generates a fifth image by performing anoise reduction process on the plurality of first images. The noisereduction unit 346 generates a sixth image by executing the noisereduction process on the plurality of second images. The noise reductionunit 346 generates a seventh image by executing a noise reductionprocess on a plurality of third images. The measurement processing unit347 calculates 3D coordinates of a plurality of points on the surface ofthe subject on the basis of the fifth image, the sixth image, and theseventh image.

Details of parts different from those of FIG. 12 will be described.Laser light emitted from the LD 38 is transmitted to a tip of theinsertion unit 20 via the fiber 85. The fiber 85 is connected to thestripe generation unit 89. The laser light (the second illuminationlight) emitted from the fiber 85 is incident on the stripe generationunit 89.

The laser light is incident on a first end In of the stripe generationunit 89. The first optical path 890 is connected to the first end In.The laser light incident on the first end In is transmitted through thefirst optical path 890. The first optical path 890 branches into asecond optical path 891 and a third optical path 892. The second opticalpath 891 and the third optical path 892 are connected to the firstoptical path 890. The laser light transmitted through the first opticalpath 890 is split into first laser light and second laser light. Thefirst laser light is incident to the second optical path 891. The secondlaser light is incident to the third optical path 892. The first laserlight incident to the second optical path 891 is transmitted through thesecond optical path 891. The second laser light incident to the thirdoptical path 892 is transmitted through the third optical path 892.

The second optical path 891 is connected to a second end Out1. The thirdoptical path 892 is connected to a third end Out2. The length of thesecond optical path 891 is equal to that of the third optical path 892.The first laser light transmitted through the second optical path 891 isemitted from the second end Out1 and is radiated to the subject. Thesecond laser light transmitted through the third optical path 892 isemitted from the third end Out2 and radiated to the subject.

The second end Out1 and the third end Out2 are separated from each otherby a predetermined width d. The first laser light emitted from thesecond end Out1 and the second laser light emitted from the third endOut2 interfere with each other. One striped pattern is formed by theinterference of laser light. The pattern can be approximately regardedto be a stripe pattern emitted from one light source. In the stripepattern, an elongated bright part and an elongated dark part arealternately arranged.

A cycle of the stripe pattern is a cycle based on the principle ofYoung's interference stripes. The cycle is a cycle based on thewavelength of the laser light guided to the first end In and the widthd. For example, when the wavelength of the laser light guided to thefirst end In becomes short in a state in which the width d is fixed, thecycle of the stripe pattern becomes short. When the wavelength of thelaser light guided to the first end In becomes long in a state in whichthe width d is fixed, the cycle of the stripe pattern becomes long. Whenthe width d is widened, i.e., when the second end Out1 and the third endOut2 are separated from each other, in a state in which a wavelengthband of the laser light guided to the first end In is fixed, the cycleof the stripe pattern becomes short. When the width d is narrowed, i.e.,when the second end Out1 and the third end Out2 are close to each other,in a state in which the wavelength band of the laser light guided to thefirst end In is fixed, the cycle of the stripe pattern becomes long.

A part of the third optical path 892 is disposed inside the phase shiftunit 893. The switching control unit 36 b disposed in the main body unit3 b supplies a current to the phase shift unit 893. The phase shift unit893 generates heat on the basis of the current supplied from theswitching control unit 36 b. A refractive index of the third opticalpath 892 is changed by the generated heat and an optical path length ofthe third optical path 892 is changed. The phase shift unit 893 shiftsthe phase of the second laser light guided to the third optical path 892on the basis of a change in a temperature. The phase of the first laserlight and the phase of the second laser light are different from eachother. The first laser light is emitted from the second end Out1 via thesecond optical path 891. The second laser light is emitted from thethird end Out2 via the third optical path 892 having a temperaturedifferent from that of the second optical path 891. The phase of thestripe of the stripe pattern is changed by interference in a state inwhich the phase of the first laser light and the phase of the secondlaser light are shifted. That is, the phase of the stripes is shifted. Amethod in which the phase shift unit 893 shifts the phase of the laserlight is not limited to a method using a change in a temperature.

The illumination control unit 33 a controls the white light source 37and the LD 38. The illumination control unit 33 a causes the white lightsource 37 and the LD 38 to generate illumination light during the secondperiod. The illumination control unit 33 a causes the white light source37 and the LD 38 to stop the generation of the illumination light duringthe third period. The illumination control unit 33 a functions as aswitching unit and a switching control unit. The illumination controlunit 33 a starts switching of the imaging condition during the thirdperiod and completes the switching of the imaging condition during thethird period. The switching control unit 36 b causes the phase shiftunit 893 to start the switching of the imaging condition during thethird period and complete the switching of the imaging condition duringthe third period.

Under the first imaging condition, the illumination control unit 33 acauses the LD 38 to be turned on and causes the white light source 37 tobe turned off. Therefore, the illumination optical system 80 a radiatesthe pattern light to the subject under the first imaging condition. Theillumination optical system 80 a does not radiate white light to thesubject under the first imaging condition. The switching control unit 36b causes the phase shift unit 893 to set the phase of the pattern lightto a first phase. Under the first imaging condition, the observationoptical system 60 a takes in the light reflected on the surface of thesubject illuminated with the pattern light.

Under the second imaging condition, the illumination control unit 33 acauses the LD 38 to be turned on and causes the white light source 37 tobe turned off. Therefore, the illumination optical system 80 a radiatesthe pattern light to the subject under the second imaging condition. Theillumination optical system 80 a does not radiate white light to thesubject under the second imaging condition. The switching control unit36 b causes the phase shift unit 893 to set the phase of the patternlight to the second phase. The second phase is different from the firstphase. Under the second imaging condition, the observation opticalsystem 60 a takes in the light reflected on the surface of the subjectilluminated with the pattern light.

Under the third imaging condition, the illumination control unit 33 acauses the LD 38 to be turned on and causes the white light source 37 tobe turned off. Therefore, the illumination optical system 80 a radiatesthe pattern light to the subject under the third imaging condition. Theillumination optical system 80 a does not radiate white light to thesubject under the third imaging condition. The switching control unit 36b causes the phase shift unit 893 to set the phase of the pattern lightto the third phase. The third phase is different from the first phaseand is different from the second phase. Under the third imagingcondition, the observation optical system 60 a takes in the lightreflected on the surface of the subject illuminated with the patternlight.

The illumination control unit 33 a causes the white light source 37 tobe turned on and causes the LD 38 to be turned off under the fourthimaging condition. Therefore, the illumination optical system 80 aradiates white light to the subject under the fourth imaging condition.The illumination optical system 80 a does not radiate the pattern lightto the subject under the fourth imaging condition. The observationoptical system 60 a takes in the light reflected on the surface of thesubject illuminated with the white light under the fourth imagingcondition.

FIG. 17 shows relationships between an operation of the imaging device22 in a measurement mode, a state of illumination, and a state of animaging condition. The operation of the imaging device 22 will bedescribed with reference to FIG. 17. Descriptions of parts that are thesame as those shown in FIG. 7 will be omitted.

A timing chart TC14 shows the operation of the imaging device 22. Cells54 of first to eighth row are sequentially reset and exposure periods ofthe cells 54 of the first to eighth rows during a frame period i aresequentially started.

When the exposure period of the cells 54 of the first row during theframe period i has been started, the white light source 37 and the LD 38are already turned off. When the exposure period of the cells 54 of thefirst row during the frame period i has been started, no imagingcondition is set.

When the exposure period of the cells 54 of the eighth row during theframe period i has been started, the turning-on of the white lightsource 37 is started. The illumination control unit 33 a causes thewhite light source 37 to be turned on by outputting a turning-on controlsignal to the white light source 37. Thereby, the illumination controlunit 33 a sets the imaging condition to the fourth imaging condition.The white light source 37 starts turning-on on the basis of the controlsignal from the illumination control unit 33 a.

When the reading period of the cells 54 of the first row during theframe period i has been started, the turning-on of the white lightsource 37 ends. The illumination control unit 33 a causes the whitelight source 37 to be turned off by outputting a turning-off controlsignal to the white light source 37. The white light source 37 is turnedoff on the basis of the control signal from the illumination controlunit 33 a. Thereby, the illumination control unit 33 a starts switchingfrom the fourth imaging condition to the first imaging condition.

When the reading period of the cells 54 of the eighth row during theframe period i has ended, the turning-on of the LD 38 is started. Theillumination control unit 33 a causes the LD 38 to be turned on byoutputting a turning-on control signal to the LD 38. The LD 38 startsturning-on on the basis of the control signal from the illuminationcontrol unit 33 a. The switching control unit 36 b causes the phase ofthe pattern light to be set to the first phase by outputting an electriccurrent to the phase shift unit 893. The phase shift unit 893 sets thephase of the pattern light to the first phase on the basis of theelectric current from the switching control unit 36 b. Thereby, theillumination control unit 33 a and the switching control unit 36 bcomplete switching from the fourth imaging condition to the firstimaging condition.

When the reading period of the cells 54 of the first row during theframe period (i+1) has been started, the turning-on of the LD 38 ends.The illumination control unit 33 a causes the LD 38 to be turned off byoutputting a turning-off control signal to the LD 38. The LD 38 isturned off on the basis of the control signal from the illuminationcontrol unit 33 a. Thereby, the illumination control unit 33 a startsswitching from the first imaging condition to the second imagingcondition.

When the reading period of the cells 54 of the eighth row during theframe period (i+1) has ended, the turning-on of the LD 38 is started.The illumination control unit 33 a causes the LD 38 to be turned on byoutputting a turning-on control signal to the LD 38. The LD 38 startsturning-on on the basis of the control signal from the illuminationcontrol unit 33 a. The switching control unit 36 b causes the phase ofthe pattern light to be set to the second phase by outputting anelectric current to the phase shift unit 893. The phase shift unit 893sets the phase of the pattern light to the second phase on the basis ofthe electric current from the switching control unit 36 b. Thereby, theillumination control unit 33 a and the switching control unit 36 bcomplete switching from the first imaging condition to the secondimaging condition.

When the reading period of the cells 54 of the first row during theframe period (i+2) has been started, the turning-on of the LD 38 ends.The illumination control unit 33 a causes the LD 38 to be turned off byoutputting a turning-off control signal to the LD 38. The LD 38 isturned off on the basis of the control signal from the illuminationcontrol unit 33 a. Thereby, the illumination control unit 33 a startsswitching from the second imaging condition to the third imagingcondition.

When the reading period of the cells 54 of the eighth row during theframe period (i+2) has ended, the turning-on of the LD 38 is started.The illumination control unit 33 a causes the LD 38 to be turned on byoutputting a turning-on control signal to the LD 38. The LD 38 startsturning-on on the basis of the control signal from the illuminationcontrol unit 33 a. The switching control unit 36 b causes the phase ofthe pattern light to be set to the third phase by outputting an electriccurrent to the phase shift unit 893. The phase shift unit 893 sets thephase of the pattern light to the third phase on the basis of theelectric current from the switching control unit 36 b. Thereby, theillumination control unit 33 a and the switching control unit 36 bcomplete switching from the second imaging condition to the thirdimaging condition.

When the reading period of the cells 54 of the first row during theframe period (i+3) has been started, the turning-on of the LD 38 ends.The illumination control unit 33 a causes the LD 38 to be turned off byoutputting a turning-off control signal to the LD 38. The LD 38 isturned off on the basis of the control signal from the illuminationcontrol unit 33 a. Thereby, the illumination control unit 33 a startsswitching from the third imaging condition to the fourth imagingcondition.

When the reading period of the cells 54 of the eighth row during theframe period (i+3) has ended, the turning-on of the white light source37 is started. The illumination control unit 33 a causes the white lightsource 37 to be turned on by outputting a turning-on control signal tothe white light source 37. The white light source 37 starts turning-onon the basis of the control signal from the illumination control unit 33a. Thereby, the illumination control unit 33 a completes switching fromthe third imaging condition to the fourth imaging condition.

The white light source 37 is turned on during the simultaneous exposureperiod in the frame period i. The LD 38 is turned on during thesimultaneous exposure period in each of the frame period (i+1), theframe period (i+2), and the frame period (i+3). The white light source37 and the LD 38 are iteratively turned on and off.

The illumination control unit 33 a and the switching control unit 36 biterate switching from the fourth imaging condition to the first imagingcondition, switching from the first imaging condition to the secondimaging condition, switching from the second imaging condition to thethird imaging condition, and switching from the third imaging conditionto the fourth imaging condition. The illumination control unit 33 a andthe switching control unit 36 b execute switching of the imagingcondition during each frame period.

During the first period T1, the imaging device 22 sequentially readspixel signals from at least some of the plurality of cells 54 row byrow. In a timing chart TC14 shown in FIG. 17, the first period T1 is aperiod from the start of the reading of the pixel signals of the cells54 of the first row to the completion of the reading of the pixelsignals of the cells 54 of the eighth row.

The white light source 37 or the LD 38 is turned on simultaneously withthe completion of reading of pixel signals during one frame period. Thatis, the white light source 37 or the LD 38 is turned on simultaneouslywith the completion of the first period T1. The white light source 37 orthe LD 38 is turned off simultaneously with the start of the reading ofthe pixel signals during one frame period. That is, the white lightsource 37 or the LD 38 is turned off simultaneously with the start ofthe first period T1.

The illumination control unit 33 a causes the white light source 37 orthe LD 38 to generate the illumination light during the second periodT2. The white light source 37 or the LD 38 is continuously turned onduring the second period T2. The second period T2 is all of a periodother than the first period T1. In the timing chart TC14 shown in FIG.17, the sum of the length of the first period T1 and the length of thesecond period T2 is the same as the length of the frame period.

The illumination control unit 33 a causes the white light source 37 orthe LD 38 to stop the generation of the illumination light during thethird period T3. The white light source 37 or the LD 38 is continuouslyturned off during the third period T3. The third period T3 is all of aperiod other than the second period T2. In the timing chart TC14 shownin FIG. 17, the third period T3 is the same as the first period T1.

The illumination control unit 33 a and the phase shift unit 893 startthe switching of the imaging condition during the third period T3 andcomplete the switching of the imaging condition during the third periodT3. The illumination control unit 33 a and the phase shift unit 893complete the switching of the imaging condition before the next secondperiod is started. That is, the illumination control unit 33 a and thephase shift unit 893 complete the switching of the imaging conditionduring the third period that is the same as the third period duringwhich the switching of the imaging condition has been started. When thethird period T3 has been started, the illumination control unit 33 astarts the switching of the imaging condition by causing the white lightsource 37 or the LD 38 to be turned off. When the third period T3 ends,the illumination control unit 33 a completes the switching of theimaging condition by causing the white light source 37 or the LD 38 tobe turned on. When the third period T3 ends, the phase shift unit 893completes the switching of the imaging condition by changing the phaseof the pattern light. Because the third period T3 is the same as thefirst period T1 in the timing chart TC14 shown in FIG. 17, theillumination control unit 33 a and the phase shift unit 893 start theswitching of the imaging condition during the first period T1 andcomplete the switching of the imaging condition during the first periodT1.

After the reading period of the cells 54 of the first row during eachframe period is started and before the reading period of the cells 54 ofthe eighth row during each frame period is completed, the phase shiftunit 893 may switch the phase of the pattern light.

FIG. 18 shows a procedure of an operation of the endoscope device 1 b.The operation of the endoscope device 1 b will be described withreference to FIG. 18. Descriptions of processing that is the same asthat shown in FIG. 14 will be omitted.

When the control unit 340 determines to execute the measurement in stepS120, the imaging device 22 generates an image of one frame and outputsthe generated image. Because the imaging condition is the observationillumination, the imaging device 22 outputs a fourth image. When thethird period is started, the illumination control unit 33 a causes thewhite light source 37 to be turned off (step S130 a).

After the step S130 a, the illumination control unit 33 a causes the LD38 to be turned on and the phase shift unit 893 sets a first phase.Thereby, the illumination control unit 33 a and the phase shift unit 893set the imaging condition to a stripe having the first phase (the firstimaging condition) (step S132 b).

After step S132 b, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging condition is astripe having the first phase, the imaging device 22 outputs a secondimage. When the third period starts, the illumination control unit 33 acauses the LD 38 to be turned off (step S145 b).

After step S145 b, the illumination control unit 33 a causes the LD 38to be turned on and the phase shift unit 893 sets the second phase. Thesecond phase is a phase obtained by shifting the first phase by 2π/3.Thereby, the illumination control unit 33 a and the phase shift unit 893set the imaging condition to a stripe having the second phase (thesecond imaging condition) (step S170).

After step S170, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging condition is astripe having the second phase, the imaging device 22 outputs the secondimage. When the third period starts, the illumination control unit 33 acauses the LD 38 to be turned off (step S175).

After step S175, the illumination control unit 33 a causes the LD 38 tobe turned on and the phase shift unit 893 sets the third phase. Thethird phase is a phase obtained by shifting the second phase by 2π/3.Thereby, the illumination control unit 33 a and the phase shift unit 893set the imaging condition to a stripe having the third phase (the thirdimaging condition) (step S180).

After step S180, the imaging device 22 generates an image of one frameand outputs the generated image. Because the imaging condition is astripe having the third phase, the imaging device 22 outputs the secondimage. When the third period starts, the illumination control unit 33 acauses the LD 38 to be turned off (step S185).

After step S185, the illumination control unit 33 a sets the imagingcondition to observation illumination (the fourth imaging condition) bycausing the white light source 37 to be turned on (step S190). Afterstep S190, the processing in step S150 is executed.

The process shown in FIG. 10 is changed to the following process. Animage set A and an image set B are input to the CPU 34. Each of theimage set A and the image set B includes one first image, one secondimage, one third image, and one fourth image.

In a motion detection process (step S200) serving as the suitabilitydetermination process, the suitability determination unit 345 calculatesa first value indicating first motion between two frames on the basis ofthe first image of the set A and the first image of the set B. In themotion detection process, the suitability determination unit 345calculates a second value indicating second motion between two framesbased on the second image of the set A and the second image of the setB. In the motion detection process, the suitability determination unit345 calculates a third value indicating second motion between two frameson the basis of the third image of the set A and the third image of theset B. In the motion detection process, the suitability determinationunit 345 calculates a fourth value indicating second motion between twoframes on the basis of the fourth image of the set A and the fourthimage of the set B. For example, the suitability determination unit 345calculates an absolute value of the difference between pixel values oftwo images for each cell 54. The suitability determination unit 345calculates a sum of absolute values of differences of pixel values ofall pixels of the image. The suitability determination unit 345calculates the first value, the second value, the third value, and thefourth value as determination parameters. The first value is a sum ofabsolute values of differences between pixel values calculated from thetwo first images. The second value is a sum of absolute values ofdifferences between pixel values calculated from the two second images.The third value is a sum of absolute values of differences of pixelvalues calculated from the two third images.

The fourth value is a sum of absolute values of differences betweenpixel values calculated from the two fourth images. The motion detectionmethod may be another method. The suitability determination method maybe another method other than motion detection.

In step S205, the suitability determination unit 345 determines whetheror not an image set is suitable for image processing of a subsequentstage. The suitability determination unit 345 determines whether or noteach determination parameter is less than a predetermined value. Whenthe suitability determination unit 345 determines that all of thedetermination parameters are less than the predetermined value, thesuitability determination unit 345 determines that the image set issuitable (has suitability) for the image processing of the subsequentstage. When the suitability determination unit 345 determines that atleast one of the determination parameters is greater than or equal tothe predetermined value, the suitability determination unit 345determines that the image set is not suitable (does not have anysuitability) for image processing of the subsequent stage.

In the noise reduction process (step S215), the noise reduction unit 346generates a first NR image (a fifth image) on the basis of the firstimage of the set A and the first image of the set B. A pixel value ofthe first NR image is an addition average of pixel values between thetwo first images. In the noise reduction process, the noise reductionunit 346 generates a second NR image (a sixth image) on the basis of thesecond image of the set A and the second image of the set B. A pixelvalue of the second NR image is an addition average of pixel valuesbetween the two second images. In the noise reduction process, the noisereduction unit 346 generates a third NR image (a seventh image) on thebasis of the third image of the set A and the third image of the set B.A pixel value of the third NR image is an addition average of pixelvalues between the two third images.

In the measurement process (step S220), the measurement processing unit347 generates a first corrected image by correcting optical distortionof the first NR image. In the measurement process, the measurementprocessing unit 347 generates a second corrected image by correctingoptical distortion of the second NR image. In the measurement process,the measurement processing unit 347 generates a third corrected image bycorrecting optical distortion of the third NR image. In the measurementprocess, the measurement processing unit 347 calculates 3D coordinatesof each point on the surface of the subject by a phase shift methodusing the first NR image, the second NR image, and the third NR image.The measurement processing unit 347 generates color 3D data. The color3D data includes color information of each pixel of the fourth image and3D coordinates corresponding to each pixel.

The visualization processing unit 343 generates a perspective projectionimage in the visualization process (step S225). The CPU 34 outputs thegenerated perspective projection image to the video processing unit 31.The video processing unit 31 outputs the perspective projection image tothe display unit 5.

In the above example, the phase shift unit 893 sets three phases. Theamount of change in the phase in switching of the imaging condition is2n/3. The phase shift unit 893 may set four or more phases. When thefour or more phases are set, the amount of change in the phase inswitching of the imaging condition is n/2 or less. When the four or morephases are set, one or more images are acquired in addition to the threeimages (the first image, the second image, and the third image)corresponding to three phases.

The suitability determination unit 345 may calculate only any one of thefirst value, the second value, the third value, and the fourth value inthe motion detection process. The suitability determination unit 345 maydetermine whether or not motion has been detected on the basis of onlyany one of the first value, the second value, the third value, and thefourth value. In the motion detection process, the suitabilitydetermination unit 345 may calculate only two of the first value, thesecond value, the third value, and the fourth value. The suitabilitydetermination unit 345 may determine whether or not motion has beendetected on the basis of only any two of the first value, the secondvalue, the third value, and the fourth value. The suitabilitydetermination unit 345 may calculate only three of the first value, thesecond value, the third value, and the fourth value in the motiondetection process. The suitability determination unit 345 may determinewhether or not motion has been detected on the basis of only any threeof the first value, the second value, the third value, and the fourthvalue.

The imaging device 22 may generate a plurality of images under any oneof the four imaging conditions and generate one image under theremaining three imaging conditions. In this case, the suitabilitydetermination unit 345 calculates any one of the first value, the secondvalue, the third value, and the fourth value on the basis of a pluralityof images generated under one imaging condition. The imaging device 22may generate a plurality of images under any two of the four imagingconditions and generate one image under the remaining two imagingconditions. In this case, the suitability determination unit 345calculates any two of the first value, the second value, the thirdvalue, and the fourth value on the basis of the plurality of imagesgenerated under the two imaging conditions. The imaging device 22 maygenerate a plurality of images under any three of the four imagingconditions and generate one image under the remaining one imagingcondition. In this case, the suitability determination unit 345calculates any three of the first value, the second value, the thirdvalue, and the fourth value on the basis of the plurality of imagesgenerated under the three imaging conditions.

The image processing unit 342 may process three or more image sets andselect a set most suitable for image processing. Even if motion isdetected between two images of two consecutive sets, there is apossibility that motion will not be detected between the other twoimages of the two consecutive sets. Thus, a probability of redoing ofimaging decreases by selecting an object to be processed from three ormore image sets. The noise reduction unit 346 can further reduce noiseby executing the noise reduction process on the basis of the three ormore images.

The white light source 37, the condenser lens 81, the light guide 82,the rod lens 83, and the diffusion lens 84 are optional. Theillumination control unit 33 a and the phase shift unit 893 may switchthe imaging condition between only the first imaging condition, thesecond imaging condition, and the third imaging condition.

A means for generating a stripe pattern on the basis of secondillumination light and a means for shifting a phase of the stripepattern may be based on a method other than interference of laser light.For example, a method of projecting light emitted from an LED array andshifting the phase of the stripe pattern by switching an turning-onpattern of the LED array may be applied to the fourth embodiment.

The endoscope device 1 b can shorten time intervals of three imagingoperations on the subject to which pattern light having three types ofphases is radiated. Also, the endoscope device 1 b can shorten a timeinterval between imaging of the subject irradiated with the white lightand imaging of the subject irradiated with the pattern light. Thus, asin the first embodiment, the quality of an image processing result isimproved. Measurement accuracy is improved in measurement process. Adeviation of color texture with respect to a reconstructed 3D shape ofthe subject becomes small. Furthermore, because the endoscope device 1 bperforms global exposure, it is possible to reduce deterioration in thequality of the image processing result due to an influence of rollingdistortion.

Second illumination light generated by the LD 38 may be laser lighthaving a wavelength other than that of blue. For example, the wavelengthof the second illumination light may be a red, green, or infraredwavelength. Also, under at least one of the first imaging condition, thesecond imaging condition, and the third imaging condition, radiation ofthe first illumination light may not be stopped.

Fifth Embodiment

FIG. 19 shows a configuration of an endoscope device 1 c according to afifth embodiment of the present invention. Descriptions of parts thatare the same as those shown in FIG. 2 will be omitted.

The main body unit 3 shown in FIG. 2 is changed to a main body unit 3 c.In the main body unit 3 c, the switching control unit 36 shown in FIG. 2is changed to a switching control unit 36 c. In the main body unit 3 c,the CPU 34 shown in FIG. 2 is changed to a CPU 34 c.

The optical adapter 21 shown in FIG. 2 is changed to an optical adapter21 c. In the optical adapter 21 c, the observation optical system 60shown in FIG. 2 is changed to an observation optical system 60 c. Theobservation optical system 60 c includes a concave lens 23, a convexlens 24, and a switching unit 25 c. The concave lens 23 is the same asthe concave lens 23 shown in FIG. 12. The convex lens 24 is the same asthe convex lens 24 shown in FIG. 12.

FIG. 20 shows a functional configuration of the CPU 34 c. Descriptionsof parts that are the same as those shown in FIG. 3 will be omitted. TheCPU 34 c does not have the visualization processing unit 343 shown inFIG. 3. The image processing unit 342 shown in FIG. 3 is changed to animage processing unit 342 c. The image processing unit 342 c includes asuitability determination unit 345, a noise reduction unit 346, and asynthesis processing unit 348.

The outline of parts different from those of FIG. 2 will be described. Afocus of the observation optical system 60 c under a first imagingcondition is different from a focus of the observation optical system 60c under a second imaging condition. The synthesis processing unit 348generates a third image by synthesizing a first image and a secondimage.

The switching control unit 36 c causes the switching unit 25 c to startswitching of the imaging condition during a third period and completethe switching of the imaging condition during the third period.

The switching control unit 36 c causes the switching unit 25 c to setthe first imaging condition during a plurality of first frame periods.The switching control unit 36 c causes the switching unit 25 c to setthe second imaging condition during a plurality of second frame periods.Each second frame period of the plurality of second frame periods isdifferent from each first frame period of the plurality of first frameperiods. The noise reduction unit 346 generates a fourth image byperforming a noise reduction process on a plurality of first images. Thenoise reduction unit 346 generates a fifth image by performing the noisereduction process on a plurality of second images. The synthesisprocessing unit 348 generates the third image by synthesizing the fourthimage and the fifth image.

Details of parts different from those of FIG. 2 will be described. Theswitching unit 25 c has a direct-acting solenoid. A mover of thesolenoid is connected to the convex lens 24. The switching control unit36 c supplies electric power to the switching unit 25 c. The switchingcontrol unit 36 c causes the switching unit 25 c to switch a position ofthe solenoid. The switching unit 25 c switches the focus of theobservation optical system 60 c by switching the position of thesolenoid. For example, a first focus under the first imaging conditionis a far point and a second focus under the second imaging condition isa near point. The first focus under the first imaging condition may be anear point and the second focus under the second imaging condition maybe a far point. The synthesis processing unit 348 synthesizes aplurality of images having different focuses to perform depth synthesis.According to the depth synthesis, a depth of field of an imageincreases.

FIG. 21 shows a procedure of the operation of the endoscope device 1 c.The operation of the endoscope device 1 c will be described withreference to FIG. 21. Descriptions of processing that is the same asthat shown in FIG. 9 will be omitted.

After step S100, the switching control unit 36 c outputs a focusswitching control signal to the switching unit 25 c. For example, theuser inputs a focus indication by operating the operation unit 4. Forexample, the focus indicated by the user is one of a first focus and asecond focus. The switching control unit 36 c outputs a control signalindicating the focus indicated by the user to the switching unit 25 c.Thereby, the switching control unit 36 c causes the switching unit 25 cto start switching of the imaging condition. The switching unit 25 cstarts focus switching on the basis of the control signal from theswitching control unit 36 c. Thereafter, the switching control unit 36 ccauses the switching unit 25 c to complete the focus switching (stepS105 c). When the imaging condition is the focus indicated by the userin step S105 c, the processing in step S105 c is unnecessary. After stepS105 c, the processing in step S110 is executed.

After step S115, the control unit 340 determines whether or not toexecute synthesis (step S120 c). For example, when the user inputs asynthesis instruction by operating the operation unit 4, the controlunit 340 determines to execute synthesis.

When the control unit 340 determines not to execute the synthesis instep S120 c, the processing in step S105 c is executed. The focus of theobservation optical system 60 c is maintained as the focus indicated bythe user until the synthesis instruction is input. Until the synthesisinstruction is input, the imaging device 22 sequentially outputs imagesand the display unit 5 sequentially updates and displays the images.

When the control unit 340 determines to execute the synthesis in stepS120 c, the switching control unit 36 c outputs a focus switchingcontrol signal to the switching unit 25 c. Thereby, the switchingcontrol unit 36 c causes the switching unit 25 c to start focusswitching. The switching unit 25 c starts switching from the focusindicated by the user to the first focus or switching from the secondfocus to the first focus on the basis of the control signal from theswitching control unit 36 c. Thereafter, the switching control unit 36 ccauses the switching unit 25 c to complete the focus switching. Thereby,the switching control unit 36 c sets the imaging condition to the firstfocus (a first imaging condition) (step S125 c). After step S125 c, theprocessing in step S130 is executed.

After step S135, the switching control unit 36 c outputs the focusswitching control signal to the switching unit 25 c. Thereby, theswitching control unit 36 c causes the switching unit 25 c to start thefocus switching. The switching unit 25 c starts switching from the firstfocus to the second focus on the basis of the control signal from theswitching control unit 36 c. Thereafter, the switching control unit 36 ccauses the switching unit 25 c to complete the focus switching. Thereby,the switching control unit 36 c sets the imaging condition to the secondfocus (a second imaging condition) (step S140 c). After step S140 c, theprocessing in step S145 is executed.

When the control unit 340 determines that the acquisition of apredetermined number of image sets has been completed in step S150, theimage processing unit 342 c executes image processing (step S155 c).After step S155 c, the processing in step S160 is executed.

FIG. 22 shows details of image processing in step S155 c. The operationof the endoscope device 1 c will be described with reference to FIG. 22.Descriptions of processing that is the same as that shown in FIG. 10will be omitted.

When the suitability determination unit 345 determines that there issuitability in step S205, the noise reduction unit 346 executes a noisereduction process. In the noise reduction process, the noise reductionunit 346 generates a first NR image (a fourth image) on the basis of afirst image of a set A and a first image of a set B. A pixel value ofthe first NR image is an addition average of pixel values between thetwo first images. In the noise reduction process, the noise reductionunit 346 generates a second NR image (a fifth image) on the basis of asecond image of the set A and a second image of the set B. A pixel valueof the second NR image is an addition average of pixel values betweenthe two second images (step S215).

After step S215, the synthesis processing unit 348 executes a synthesisprocess. The synthesis processing unit 348 generates a first contrastmap and a second contrast map in the synthesis process. The firstcontrast map includes a first value of contrast at each pixel of thefirst NR image. The second contrast map includes a second value ofcontrast at each pixel of the second NR image. The synthesis processingunit 348 compares the first value with the second value for each pixelin the synthesis process. The synthesis processing unit 348 selects apixel value of the first NR image in a pixel in which the first value isgreater than the second value. The synthesis processing unit 348 selectsthe pixel value of the second NR image in a pixel in which the secondvalue is greater than the first value. In the synthesis process, thesynthesis processing unit 348 generates a third image including theselected pixel values as a synthesized image (step S240).

After step S240, the display processing unit 341 causes the display unit5 to display the synthesized image generated in step S240. The displayunit 5 displays the synthesized image (step S230 c). After step S230 c,the processing in step S235 is executed.

When the noise reduction process in step S215 is not executed, thesynthesis processing unit 348 executes the synthesis process using oneimage set. The synthesis processing unit 348 generates a third image bysynthesizing the first image corresponding to the first focus and thesecond image corresponding to the second focus. One image set may beselected from a plurality of image sets and the synthesis processingunit 348 may execute the synthesis process using the selected image set.

The synthesis processing unit 348 may synthesize three or more images.

Focuses of the observation optical system 60 c when the three or moreimages are generated are different from each other.

The endoscope device 1 c can shorten a time interval between imaging inthe first focus and imaging in the second focus. Thus, as in the firstembodiment, the quality of the image processing result is improved. Inthe image, the occurrence of an unnatural edge due to movement of thesubject while the focus is being changed or movement of a tip of aninsertion unit 20 is suppressed. Furthermore, because the endoscopedevice 1 c performs global exposure, it is possible to reduce thedeterioration of the quality of the image processing result due to aninfluence of rolling distortion. Therefore, the quality of a depthsynthesis result is improved.

Sixth Embodiment

FIG. 23 shows a configuration of an endoscope device 1 d according to asixth embodiment of the present invention. Descriptions of parts thatare the same as those shown in FIG. 2 will be omitted.

The main body unit 3 shown in FIG. 2 is changed to a main body unit 3 d.The main body unit 3 d does not have the switching control unit 36. Inthe main body unit 3 d, the illumination control unit 33 shown in FIG. 2is changed to an illumination control unit 33 d. In the main body unit 3d, the CPU 34 shown in FIG. 2 is changed to a CPU 34 c. The CPU 34 c isthe same as the CPU 34 c shown in FIG. 19.

The optical adapter 21 shown in FIG. 2 is changed to an optical adapter21 d. In the optical adapter 21 d, the observation optical system 60shown in FIG. 2 is changed to an observation optical system 60 a. Theobservation optical system 60 a is the same as the observation opticalsystem 60 a shown in FIG. 12.

The outline of parts different from those of FIG. 2 will be described.The amount of light of the white light source 37 under a first imagingcondition is different from the amount of light of the white lightsource 37 under a second imaging condition. The synthesis processingunit 348 of the image processing unit 342 c generates a third image bysynthesizing a first image and a second image.

The illumination control unit 33 d functions as a switching unit and aswitching control unit. The illumination control unit 33 d startsswitching of the imaging condition during the third period and completesthe switching of the imaging condition during the third period. Theillumination control unit 33 d completes the switching of the imagingcondition before the next second period is started. That is, theillumination control unit 33 d completes the switching of the imagingconditions during the third period that is the same as the third periodduring which the switching of the imaging condition has been started.

The illumination control unit 33 d sets a first imaging condition duringa plurality of first frame periods. The illumination control unit 33 dsets a second imaging condition during a plurality of second frameperiods. Each second frame period of the plurality of second frameperiods is different from each first frame period of the plurality offirst frame periods. The noise reduction unit 346 generates a fourthimage by executing a noise reduction process on the plurality of firstimages. The noise reduction unit 346 generates a fifth image byexecuting the noise reduction process on the plurality of second images.The synthesis processing unit 348 generates the third image bysynthesizing the fourth image and the fifth image.

Details of parts different from those of FIG. 2 will be described. Theillumination control unit 33 d switches the imaging condition byswitching the amount of light of the white light source 37. Thebrightness of illumination light under the first imaging condition isdifferent from the brightness of illumination light under the secondimaging condition. Thus, an exposure amount of the cell 54 under thefirst imaging condition is different from an exposure amount of the cell54 under the second imaging condition. The synthesis processing unit 348executes high dynamic range (HDR) synthesis by synthesizing a pluralityof images having different amounts of exposure. The HDR synthesis widensa dynamic range of the image.

FIG. 24 shows a procedure of the operation of the endoscope device 1 d.The operation of the endoscope device 1 d will be described withreference to FIG. 24. Descriptions of processing that is the same asthat shown in FIG. 21 will be omitted.

After step S100, the illumination control unit 33 d sets the amount oflight of the white light source 37 to a predetermined suitable amount oflight. Thereby, the illumination control unit 33 d sets the imagingcondition to a first exposure amount (step S105 d). After step S105 d,the processing in step S110 is executed.

After step S130, the illumination control unit 33 d sets the amount oflight of the white light source 37 to ¼ of a suitable amount of light.Thereby, the illumination control unit 33 d sets the imaging conditionto a second exposure amount (the second imaging condition) (step S132d). After step S132 d, the processing in step S135 is executed.

After step S145, the illumination control unit 33 d sets the amount oflight of the white light source 37 to four times the suitable amount oflight. Thereby, the illumination control unit 33 d sets the imagingcondition to a third exposure amount (the third imaging condition) (stepS170 d).

After step S170 d, the imaging device 22 generates an image of one frameand outputs the generated image (step S175 d).

After step S175 d, the illumination control unit 33 d sets the amount oflight of the white light source 37 to a predetermined suitable amount oflight. Thereby, the illumination control unit 33 d sets the imagingcondition to the first exposure amount (the first imaging condition)(step S180 d). After step S180 d, the processing in step S150 isexecuted.

The image processing in step S155 c includes the processing shown inFIG. 22. Parts different from the image processing in the fifthembodiment will be described.

The image set A and the image set B are input to the CPU 34 c. Each ofthe image set A and the image set B includes one first exposure image,one second exposure image, and one third exposure image. The firstexposure image is an image generated when the imaging condition is thefirst exposure amount. The second exposure image is an image generatedwhen the imaging condition is the second exposure amount. The thirdexposure image is an image generated when the imaging condition is thethird exposure amount.

In the motion detection process (step S200), the suitabilitydetermination unit 345 calculates a first value indicating first motionbetween two frames on the basis of the first exposure image of the set Aand the first exposure image of the set B. In the motion detectionprocess, the suitability determination unit 345 calculates a secondvalue indicating second motion between two frames on the basis of thesecond exposure image of the set A and the second exposure image of theset B. In the motion detection process, the suitability determinationunit 345 calculates a third value indicating third motion between twoframes on the basis of the third exposure image of the set A and thethird exposure image of the set B. For example, the suitabilitydetermination unit 345 calculates an absolute value of the differencebetween the pixel values in each pixel between two images for each pixelof the image. The suitability determination unit 345 calculates a sum ofabsolute values of differences of pixel values in all the cells 54. Thefirst value is a sum of absolute values of differences between pixelvalues calculated from the two first exposure images. The second valueis a sum of absolute values of differences between pixel valuescalculated from the two second exposure images. The third value is a sumof absolute values of differences between pixel values calculated fromthe two third exposure images. The motion detection method may beanother method.

In step S205, the suitability determination unit 345 determines whetheror not motion has been detected. The suitability determination unit 345determines whether or not each of the first value, the second value, andthe third value is less than a predetermined value. When the suitabilitydetermination unit 345 determines that all of the first value, thesecond value, and the third value are less than the predetermined value,the suitability determination unit 345 determines that no motion hasbeen detected (that no motion has occurred or that there is no motion).When the suitability determination unit 345 determines that at least oneof the first value, the second value, and the third value is greaterthan or equal to the predetermined value, the suitability determinationunit 345 determines that motion has been detected (that motion hasoccurred or that there is motion).

In the noise reduction process (step S215), the noise reduction unit 346generates a first NR image (a fourth image) on the basis of the firstexposure image of the set A and the first exposure image of the set B. Apixel value of the first NR image is an addition average of pixel valuesbetween the two first exposure images. In the noise reduction process,the noise reduction unit 346 generates a second NR image (a fifth image)on the basis of the second exposure image of the set A and the secondexposure image of the set B. A pixel value of the second NR image is anaddition average of pixel values between the two second exposure images.In the noise reduction process, the noise reduction unit 346 generates athird NR image (a sixth image) on the basis of the third exposure imageof the set A and the third exposure image of the set B. A pixel value ofthe third NR image is an addition average of pixel values between thetwo third exposure images.

After step S215, the synthesis processing unit 348 executes a synthesisprocess. In the synthesis process, the synthesis processing unit 348generates a third image as a synthesized image by synthesizing the firstNR image, the second NR image, and the third NR image (step S240).

In the synthesized image, whiteout and blackout are suppressed. When thesynthesized image is displayed in step S230 c, a dark place within avisual field and a bright place within the visual field are displayedwith appropriate brightness.

When the noise reduction process in step S215 is not executed, thesynthesis processing unit 348 executes the synthesis process using oneimage set. The synthesis processing unit 348 generates the third imageby synthesizing the first exposure image corresponding to the firstexposure amount, the second exposure image corresponding to the secondexposure amount, and the third exposure image corresponding to the thirdexposure amount. One image set may be selected from a plurality of imagesets and the synthesis processing unit 348 may execute the synthesisprocess using the selected image set.

The suitability determination unit 345 may calculate only any one of thefirst value, the second value, and the third value in the motiondetection process. The suitability determination unit 345 may determinewhether or not motion has been detected on the basis of only any one ofthe first value, the second value, and the third value. The suitabilitydetermination unit 345 may calculate only any two of the first value,the second value, and the third value in the motion detection process.The suitability determination unit 345 may determine whether or notmotion has been detected on the basis of only on any two of the firstvalue, the second value, and the third value.

The imaging device 22 may generate a plurality of images under any oneof three imaging conditions and generate one image under the remainingtwo imaging conditions. In this case, the suitability determination unit345 calculates any one of the first value, the second value, and thethird value on the basis of the plurality of images generated under theone imaging condition. The imaging device 22 may generate a plurality ofimages under any two of the three imaging conditions and generate oneimage under the remaining one imaging condition. In this case, thesuitability determination unit 345 calculates any two of the firstvalue, the second value, and the third value on the basis of theplurality of images generated under the two imaging conditions.

The synthesis processing unit 348 may synthesize the two images. Amountsof light of the white light source 37 when the two images are generatedare different from each other. The synthesis processing unit 348 maysynthesize four or more images. Amounts of light of the white lightsource 37 when the four or more images are generated are different fromeach other.

The endoscope device 1 d can shorten time intervals between imaging inthe first exposure amount, imaging in the second exposure amount andimaging in the third exposure amount. Thus, as in the first embodiment,the quality of the image processing result is improved. In an image, theoccurrence of an unnatural edge due to the motion of the subject whilethe exposure amount is being changed or the movement of a tip of aninsertion unit 20 is suppressed. Furthermore, because the endoscopedevice 1 d performs global exposure, it is possible to reducedeterioration in the quality of the image processing result due to aninfluence of rolling distortion. Thus, the quality of an HDR synthesisresult is improved.

Seventh Embodiment

FIG. 25 shows a configuration of an endoscope device 1 e according to aseventh embodiment of the present invention. Descriptions of parts thatare the same as those shown in FIG. 2 will be omitted.

The main body unit 3 shown in FIG. 2 is changed to a main body unit 3 e.In the main body unit 3 e, the CPU 34 shown in FIG. 2 is changed to aCPU 34 c. The CPU 34 c is the same as the CPU 34 c shown in FIG. 19.

The outline of parts different from those of FIG. 2 will be described.The observation optical system 60 includes a first optical system, asecond optical system, and a switching unit 25. A concave lens 23 a, aconvex lens 24 a, and an image forming optical system 26 are the firstoptical system. The first optical system is disposed on an optical frontside (a subject side) of the imaging device 22 and forms a first opticalimage on the imaging device 22 based on light from the subject. Aconcave lens 23 b, a convex lens 24 b, and an image forming opticalsystem 26 are the second optical system. The second optical system isdisposed on an optical front side (a subject side) of the imaging device22 and forms a second optical image on the imaging device 22 based onlight from the subject. The switching unit 25 selects either one of thefirst optical system and the second optical system and causes onlyeither one of the first optical image and the second optical image to beformed on the imaging device 22.

A visual field of the first optical system and a visual field of thesecond optical system have a common region. The switching control unit36 switches an optical image formed on the imaging device 22 bycontrolling the switching unit 25. Under a first imaging condition, thefirst optical image is formed on the imaging device 22. Under a secondimaging condition, the second optical image is formed on the imagingdevice 22. The synthesis processing unit 348 aligns a region of a firstimage corresponding to the common region and a region of a second imagecorresponding to the common region and generates a third image bysynthesizing the first image and the second image.

The switching control unit 36 causes the switching unit 25 to set afirst imaging condition during a plurality of first frame periods. Theswitching control unit 36 causes the switching unit 25 to set a secondimaging condition during a plurality of second frame periods. Eachsecond frame period of the plurality of second frame periods isdifferent from each first frame period of the plurality of first frameperiods. The noise reduction unit 346 generates a fourth image byperforming a noise reduction process on a plurality of first images. Thenoise reduction unit 346 generates a fifth image by executing the noisereduction process on a plurality of second images. The synthesisprocessing unit 348 generates the third image by synthesizing the fourthimage and the fifth image.

Details of parts different from those of FIG. 2 will be described. Thefirst optical system and the second optical system have parallaxes withrespect to each other. Thus, the visual field of the first opticalsystem and the visual field of the second optical system are differentfrom each other. An optical axis of the first optical system and anoptical axis of the second optical system are substantially parallel toeach other. A part of the visual field of the first optical system isthe same as a part of the visual field of the second optical system. Thevisual field of the first optical system includes a first region and acommon region. The visual field of the second optical system includes asecond region and a common region. The second region is different fromthe first region.

The synthesis processing unit 348 executes panorama synthesis bysynthesizing a plurality of images having visual fields different fromeach other. In the panorama synthesis, the synthesis processing unit 348synthesizes a plurality of images after aligning common regions so thatthe common regions of images are included. The panorama synthesis widensthe visual field in the image.

The endoscope device 1 e executes the process shown in FIG. 9. The imageprocessing in step S155 shown in FIG. 9 includes the process shown inFIG. 22. Parts different from the image processing in the firstembodiment will be described.

In the synthesis process (step S240), the synthesis processing unit 348generates a third image as a synthesized image by synthesizing the firstNR image and the second NR image. In this process, the common regions ofthe images are aligned and the common region of the first NR imageoverlaps the common region of the second NR image.

When the noise reduction process in step S215 is not executed, thesynthesis processing unit 348 executes a synthesis process using oneimage set. The synthesis processing unit 348 generates a third image bysynthesizing the first image corresponding to the first optical systemand the second image corresponding to the second optical system. Oneimage set may be selected from a plurality of image sets and thesynthesis processing unit 348 may execute the synthesis process usingthe selected image set.

The optical axis of the first optical system and the optical axis of thesecond optical system may not be parallel to each other. For example,the optical axis of the first optical system may extend toward anoptical front side of the imaging device 22 and the first optical systemmay have a visual field of 120 degrees in left and right directions. Theoptical axis of the second optical system may be orthogonal to theoptical axis of the first optical system and the second optical systemmay have a visual field of 120 degrees in the left and right directions.In this case, a region of 15 degrees on the right side in the visualfield of the first optical system and a region of 15 degrees on the leftside in the visual field of the second optical system are the commonregions.

The endoscope device 1 e can shorten a time interval between imaging inthe first visual field and imaging in the second visual field. Thus, asin the first embodiment, the quality of the image processing result isimproved. In an image, the occurrence of an unnatural edge due to motionof the subject while the visual field is being changed or movement of atip of an insertion unit 20 is suppressed. Furthermore, because theendoscope device 1 e performs global exposure, it is possible to reducethe deterioration of the quality of the image processing result due toan influence of rolling distortion. Thus, the quality of the result ofthe panorama synthesis is improved.

(Supplement)

According to an aspect of the present invention, a method of operatingan endoscope device including a first step, a second step, and a thirdstep is provided. The endoscope device includes a light source, anillumination optical system, an observation optical system, an imagingdevice, a switching unit, and a control unit. The light source generatesillumination light. The illumination optical system radiates theillumination light to a subject. The observation optical system forms anoptical image of the subject. The imaging device has a plurality ofpixels disposed in a matrix, and images the subject. The imaging devicesequentially reads pixel signals from at least some of the plurality ofpixels row by row during a first period. The imaging device generates animage of the subject during each frame period of a plurality of frameperiods on the basis of the pixel signals read from at least some of theplurality of pixels. The pixel signals are generated on the basis of theoptical image of the subject. The switching unit performs switchingbetween a plurality of imaging conditions so that the imaging deviceimages the subject. In the first step, the control unit causes the lightsource to generate the illumination light during a second period. Thesecond period is at least a part of a period other than the firstperiod. In the second step, the control unit causes the light source tostop the generation of the illumination light during a third period. Thethird period is all of a period other than the second period andincludes the first period. The second period and the third period arealternately iterated. In the third step, the control unit causes theswitching unit to start switching of the imaging condition during thethird period and complete the switching of the imaging condition duringthe third period.

According to an aspect of the present invention, a program for causing aprocessor of an endoscope device to execute a first step, a second step,and a third step is provided. The endoscope device includes the lightsource, the illumination optical system, the observation optical system,the imaging device, the switching unit, and the processor. In the firststep, the processor causes the light source to generate the illuminationlight during a second period. The second period is at least a part of aperiod other than the first period. In the second step, the processorcauses the light source to stop the generation of the illumination lightduring a third period. The third period is all of a period other thanthe second period and incudes the first period. The second period andthe third period are alternately iterated. In the third step, theprocessor causes the switching unit to start switching of the imagingcondition during the third period and complete the switching of theimaging condition during the third period.

According to an aspect of the present invention, a computer-readablenon-transitory recording medium recording a program for causing aprocessor of an endoscope device to execute a first step, a second step,and a third step is provided. The endoscope device includes the lightsource, the illumination optical system, the observation optical system,the imaging device, the switching unit, and the processor. In the firststep, the processor causes the light source to generate the illuminationlight during a second period. The second period is at least a part of aperiod other than the first period. In the second step, the processorcauses the light source to stop the generation of the illumination lightduring a third period. The third period is all of a period other thanthe second period and incudes the first period. The second period andthe third period are alternately iterated. In the third step, theprocessor causes the switching unit to start switching of the imagingcondition during the third period and complete the switching of theimaging condition during the third period.

While preferred embodiments of the invention have been described andshown above, it should be understood that these are exemplars of theinvention and are not to be considered as limiting. Additions,omissions, substitutions, and other modifications can be made withoutdeparting from the spirit or scope of the present invention.Accordingly, the invention is not to be considered as being limited bythe foregoing description, and is only limited by the scope of theappended claims.

What is claimed is:
 1. An endoscope device comprising: a light sourceconfigured to generate illumination light; an illumination opticalsystem configured to radiate the illumination light to a subject; anobservation optical system configured to form an optical image of thesubject; an imaging device having a plurality of pixels disposed in amatrix and configured to image the subject, sequentially read pixelsignals from at least some of the plurality of pixels row by row duringa first period, and generate an image data of the subject during eachframe period of a plurality of frame periods on the basis of the pixelsignals read from at least some of the plurality of pixels, the pixelsignals being generated on the basis of the optical image of thesubject; a switching unit configured to perform switching between aplurality of imaging conditions so that the imaging device images thesubject; and a control unit, wherein the control unit causes the lightsource to generate the illumination light during a second period, thesecond period being at least a part of a period other than the firstperiod, the control unit causes the light source to stop the generationof the illumination light during a third period, the third period is allof a period other than the second period and includes the first period,and the second period and the third period are alternately iterated, andthe control unit causes the switching unit to start switching of theimaging condition during the third period and complete the switching ofthe imaging condition during the third period.
 2. The endoscope deviceaccording to claim 1, wherein the imaging device reads the pixel signalsin a time period that is less than or equal to half a length of eachframe period of the plurality of frame periods during the first period.3. The endoscope device according to claim 1, further comprising animage processing unit configured to execute image processing on aplurality of images generated during the plurality of frame periods,wherein the plurality of imaging conditions include a first imagingcondition and a second imaging condition, the first imaging conditionand the second imaging condition are different from each other, theimaging device generates a first image of the subject by imaging thesubject under the first imaging condition, the imaging device generatesa second image of the subject by imaging the subject under the secondimaging condition, and the image processing unit processes the firstimage and the second image.
 4. The endoscope device according to claim3, wherein the imaging device generates a plurality of at least one offirst images and second images, when the imaging device generates theplurality of first images, the image processing unit calculates a valueindicating whether or not the plurality of first images are suitable forimage processing, and when the imaging device generates the plurality ofsecond images, the image processing unit calculates a value indicatingwhether or not the plurality of second images are suitable for imageprocessing.
 5. The endoscope device according to claim 3, wherein theimaging device generates a plurality of first images by imaging thesubject under the first imaging condition, the imaging device generatesa plurality of second images by imaging the subject under the secondimaging condition, the image processing unit generates a third image byexecuting a noise reduction process on the plurality of first images,the image processing unit generates a fourth image by executing thenoise reduction process on the plurality of second images, and the imageprocessing unit executes a process different from the noise reductionprocess on the third image and the fourth image.
 6. The endoscope deviceaccording to claim 3, wherein the image processing unit calculatesthree-dimensional (3D) coordinates of at least one point on a surface ofthe subject on the basis of the first image and the second image.
 7. Theendoscope device according to claim 5, wherein the image processing unitcalculates 3D coordinates of at least one point on a surface of thesubject on the basis of the third image and the fourth image.
 8. Theendoscope device according to claim 6, further comprising a datageneration unit, wherein the illumination light is white light, theimaging device includes a red color filter, a green color filter, and ablue color filter, the imaging device generates a color image as theimage of the subject, the color image has information indicating each ofbrightness of red, brightness of green, and brightness of blue, and thedata generation unit generates data in which 3D coordinates of aplurality of points on the surface of the subject are associated withthe information corresponding to the plurality of points.
 9. Theendoscope device according to claim 6, wherein the observation opticalsystem includes a first optical system and a second optical system, thefirst optical system and the second optical system are disposed on anoptical front side of the imaging device, the first optical system formsa first optical image of the subject corresponding to a first viewpointon the imaging device, the second optical system forms a second opticalimage of the subject corresponding to a second viewpoint different fromthe first viewpoint on the imaging device, the switching unit causeslight that passes through the first optical system to be incident on theimaging device and blocks light that passes through the second opticalsystem under the first imaging condition, the switching unit causeslight that passes through the second optical system to be incident onthe imaging device and blocks light that passes through the firstoptical system under the second imaging condition, the control unitswitches the optical image to be formed on the imaging device betweenthe first optical image and the second optical image by controlling theswitching unit, and the image processing unit calculates the 3Dcoordinates using a passive stereo method on the basis of the firstimage corresponding to the first optical image and the second imagecorresponding to the second optical image.
 10. The endoscope deviceaccording to claim 1, further comprising: an image processing unitconfigured to execute image processing on a plurality of images duringeach frame period of the plurality of frame periods; a data generationunit; a first light source serving as the light source; and a secondlight source serving as the light source, wherein the illumination lightincludes first illumination light and second illumination light, thefirst light source generates white light serving as the firstillumination light, the second light source generates the secondillumination light, the illumination optical system includes a patterngeneration unit configured to give a spatial pattern including a brightpart and a dark part to the second illumination light, the illuminationoptical system radiates the second illumination light to which thepattern is given to the subject, the plurality of imaging conditionsinclude a first imaging condition and a second imaging condition, underthe first imaging condition, the first illumination light is radiated tothe subject and radiation of the second illumination light to thesubject is stopped, under the second imaging condition, the secondillumination light is radiated to the subject and radiation of the firstillumination light to the subject is stopped, the imaging devicegenerates a first image of the subject by imaging the subject under thefirst imaging condition, the imaging device generates a second image ofthe subject by imaging the subject under the second imaging condition,the image processing unit calculates 3D coordinates of a plurality ofpoints on a surface of the subject using an active stereo method on thebasis of the second image, the imaging device includes a red colorfilter, a green color filter, and a blue color filter, the imagingdevice generates a color image as the first image, the color image hasinformation indicating each of brightness of red, brightness of green,and brightness of blue, and the data generation unit generates data inwhich the 3D coordinates of the plurality of points are associated withthe information corresponding to the plurality of points.
 11. Theendoscope device according to claim 10, wherein the imaging devicegenerates a plurality of at least one of first images and second images,when the imaging device generates the plurality of first images, theimage processing unit calculates a value indicating whether or not theplurality of first images are suitable for image processing, and whenthe imaging device generates the plurality of second images, the imageprocessing unit calculates a value indicating whether or not theplurality of second images are suitable for image processing.
 12. Theendoscope device according to claim 10, wherein the imaging devicegenerates a plurality of second images by imaging the subject under thesecond imaging condition, the image processing unit generates a thirdimage by executing a noise reduction process on the plurality of secondimages, and the image processing unit calculates the 3D coordinates ofthe plurality of points on the basis of the third image.
 13. Theendoscope device according to claim 3, wherein the illumination opticalsystem includes a pattern generation unit configured to give a spatialpattern including a bright part and a dark part to the illuminationlight, the plurality of imaging conditions further include a thirdimaging condition, the imaging device generates a third image of thesubject by imaging the subject under the third imaging condition, theswitching unit performs switching between the first imaging condition,the second imaging condition, and the third imaging condition by causinga phase of the pattern of the illumination light to be shifted, apattern phase under the first imaging condition, a pattern phase underthe second imaging condition, and a pattern phase under the thirdimaging condition are different from one another, and the imageprocessing unit calculates 3D coordinates of a plurality of points on asurface of the subject using a phase shift method on the basis of atleast the first image, the second image, and the third image.
 14. Theendoscope device according to claim 13, further comprising: a datageneration unit; a first light source serving as the light source; and asecond light source serving as the light source, wherein theillumination light includes first illumination light and secondillumination light, the first light source generates white light servingas the first illumination light, the second light source generates thesecond illumination light, the pattern generation unit gives the patternto the second illumination light, the plurality of imaging conditionsfurther include a fourth imaging condition, under the first imagingcondition, the second imaging condition, and the third imagingcondition, the second illumination light is radiated to the subject andradiation of the first illumination light to the subject is stopped,under the fourth imaging condition, the first illumination light isradiated to the subject and radiation of the second illumination lightto the subject is stopped, the imaging device generates a fourth imageof the subject by imaging the subject under the fourth imagingcondition, the imaging device includes a red color filter, a green colorfilter, and a blue color filter, the imaging device generates a colorimage as the fourth image, the color image has information indicatingeach of brightness of red, brightness of green, and brightness of blue,and the data generation unit generates data in which the 3D coordinatesof the plurality of points on the surface of the subject are associatedwith the information corresponding to the plurality of points.
 15. Theendoscope device according to claim 14, wherein the imaging devicegenerates a plurality of at least one of first images, second images,third images, and fourth images, when the imaging device generates theplurality of first images, the image processing unit calculates a valueindicating whether or not the plurality of first images are suitable forimage processing, when the imaging device generates the plurality ofsecond images, the image processing unit calculates a value indicatingwhether or not the plurality of second images are suitable for imageprocessing, when the imaging device generates the plurality of thirdimages, the image processing unit calculates a value indicating whetheror not the plurality of third images are suitable for image processing,and when the imaging device generates the plurality of fourth images,the image processing unit calculates a value indicating whether or notthe plurality of fourth images are suitable for image processing. 16.The endoscope device according to claim 13, wherein the control unitcauses the switching unit to set the first imaging condition during aplurality of first frame periods, the control unit causes the switchingunit to set the second imaging condition during a plurality of secondframe periods and each second frame period of the plurality of secondframe periods is different from each first frame period of the pluralityof first frame periods, the control unit causes the switching unit toset the third imaging condition during a plurality of third frameperiods and each third frame period of the plurality of third frameperiods is different from each first frame period of the plurality offirst frame periods and is different from each of second frame period ofthe plurality of second frame periods, the image processing unitgenerates a fifth image by executing a noise reduction process on aplurality of first images, the image processing unit generates a sixthimage by executing the noise reduction process on a plurality of secondimages, the image processing unit generates a seventh image by executingthe noise reduction process on a plurality of third images, and theimage processing unit calculates the 3D coordinates of the plurality ofpoints on the basis of the fifth image, the sixth image, and the seventhimage.
 17. The endoscope device according to claim 3, wherein a focus ofthe observation optical system under the first imaging condition isdifferent from a focus of the observation optical system under thesecond imaging condition, and the image processing unit generates athird image by synthesizing the first image and the second image. 18.The endoscope device according to claim 3, wherein the amount of lightof the light source under the first imaging condition is different fromthe amount of light of the light source under the second imagingcondition, and the image processing unit generates a third image bysynthesizing the first image and the second image.
 19. The endoscopedevice according to claim 3, wherein the observation optical systemincludes a first optical system disposed on an optical front side of theimaging device and configured to form a first optical image of thesubject on the imaging device, a second optical system disposed on anoptical front side of the imaging device and configured to form a secondoptical image of the subject on the imaging device, and the switchingunit configured to select either one of the first optical system and thesecond optical system and cause only either one of the first opticalimage and the second optical image to be formed on the imaging device, avisual field of the first optical system and a visual field of thesecond optical system have a common region, the switching control unitswitches the optical image formed on the imaging device by controllingthe switching unit, the first optical image is formed on the imagingdevice under the first imaging condition, the second optical image isformed on the imaging device under the second imaging condition, and theimage processing unit aligns a region of the first image correspondingto the common region and a region of the second image corresponding tothe common region and generates a third image by synthesizing the firstimage and the second image.
 20. The endoscope device according to claim3, wherein a focus of the observation optical system under the firstimaging condition is different from a focus of the observation opticalsystem under the second imaging condition, the control unit causes theswitching unit to set the first imaging condition during a plurality offirst frame periods, the control unit causes the switching unit to setthe second imaging condition during a plurality of second frame periodsand each second frame period of the plurality of second frame periods isdifferent from each first frame period of the plurality of first frameperiods, the image processing unit generates a fourth image by executinga noise reduction process on a plurality of first images, the imageprocessing unit generates a fifth image by executing the noise reductionprocess on a plurality of second images, and the image processing unitgenerates the third image by synthesizing the fourth image and the fifthimage.